代码改变世界

CentOS7的udev的绑定规则

2020-09-08 16:54  AlfredZhao  阅读(3845)  评论(0编辑  收藏  举报

客户一套RAC环境是华为的存储,共享盘是/dev/sd*,咋一看还怀疑是没有进行多路径配置,实际和主机工程师是已经配置好的,我们使用upadmin show vlun命令可以查看到:

[root@xxdb01 ~]# upadmin show vlun
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Vlun ID  Disk          Name                      Lun WWN               Status  Capacity  Ctrl(Own/Work)    Array Name    Dev Lun ID  No. of Paths(Available/Total) 
    0     sdb   LUN_Oracle_400G_0000  6acb3b510041191b0de7bcdd0000000f  Normal  400.00GB      0D/0D       Huawei.18500V5      15                   8/8              
    1     sdc   LUN_Oracle_400G_0001  6acb3b510041191b0de7be3900000010  Normal  400.00GB      0A/0A       Huawei.18500V5      16                   8/8              
    2     sdd   LUN_Oracle_400G_0002  6acb3b510041191b0de7bec100000011  Normal  400.00GB      0B/0B       Huawei.18500V5      17                   8/8              
    3     sde   LUN_Oracle_400G_0003  6acb3b510041191b0de7bfc900000012  Normal  400.00GB      0C/0C       Huawei.18500V5      18                   8/8              
    4     sdf   LUN_Oracle_400G_0004  6acb3b510041191b0de7c03f00000013  Normal  400.00GB      0D/0D       Huawei.18500V5      19                   8/8              
    5     sdg   LUN_Oracle_400G_0005  6acb3b510041191b0de7c09e00000014  Normal  400.00GB      0A/0A       Huawei.18500V5      20                   8/8              
    6     sdh   LUN_Oracle_400G_0006  6acb3b510041191b0de7c0e900000015  Normal  400.00GB      0B/0B       Huawei.18500V5      21                   8/8              
    7     sdi   LUN_Oracle_400G_0007  6acb3b510041191b0de7c12e00000016  Normal  400.00GB      0C/0C       Huawei.18500V5      22                   8/8              
    8     sdj    LUN_Oracle_5G_0003   6acb3b510041191b0de893a0000000da  Normal   5.00GB       0C/0C       Huawei.18500V5     218                   8/8              
    9     sdk    LUN_Oracle_5G_0004   6acb3b510041191b0de8941e000000db  Normal   5.00GB       0D/0D       Huawei.18500V5     219                   8/8              
   10     sdl    LUN_Oracle_5G_0005   6acb3b510041191b0de894a4000000dc  Normal   5.00GB       0A/0A       Huawei.18500V5     220                   8/8              
   11     sdm   LUN_Oracle_100G_0002  6acb3b510041191b0de80f1f00000039  Normal  100.00GB      0B/0B       Huawei.18500V5      57                   8/8              
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@xxdb01 ~]# 

实际上直接使用这些盘也是可以的,但是考虑到规范,参考之前客户udev绑定规则规范:

--not available
KERNEL=="sd*",BUS=="scsi",PROGRAM=="/sbin/scsi_id  i --whitelisted  --device=/dev/$name",RESULT=="36000c29b263ed2452f80e9848bdf2fa5",NAME="asm-2g-2fa5-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"

可以看到别名的命名方式是:asm-lunsize-id后四位-磁盘组名+编号。这样以后遇到加盘/删盘这类操作,就能快速帮助DBA确认。
不过因为上面这个udev语法是RHEL 6的,在CentOS 7并不适用,换成7对应的语法就是:

--ok!
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-400g-000f-data1",OWNER="grid",GROUP="asmadmin",MODE="0660"

考虑到盘相对多,一个个写既费时又容易出错,想起早些年间装RAC时,经常会参考maclean的一个方法,就是把这个工作写一个脚本:
vi /u01/asmdisk.sh

for i in b c d e f g h i j k l m;
do
echo "KERNEL==\"sd*\",SUBSYSTEM==\"block\",PROGRAM==\"/lib/udev/scsi_id -g -u -d /dev/\$name\",RESULT==\"`/lib/udev/scsi_id -g -u -d /dev/sd$i`\",SYMLINK+=\"asm-5g-xxxx-grid1\",OWNER=\"grid\",GROUP=\"asmadmin\",MODE=\"0660\""
done

执行脚本:sh /u01/asmdisk.sh ,结果为:

--script-result
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7be3900000010",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bec100000011",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bfc900000012",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c03f00000013",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c09e00000014",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c0e900000015",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c12e00000016",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de893a0000000da",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de8941e000000db",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de894a4000000dc",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de80f1f00000039",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"

使用带列编辑的文本编辑器快速修改如下,然后复制到/etc/udev/rules.d/99-oracle-asmdevices.rules配置文件中:

--modify
[root@xxdb01 ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules 
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-400g-000f-data1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7be3900000010",SYMLINK+="asm-400g-0010-data2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bec100000011",SYMLINK+="asm-400g-0011-data3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bfc900000012",SYMLINK+="asm-400g-0012-data4",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c03f00000013",SYMLINK+="asm-400g-0013-data5",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c09e00000014",SYMLINK+="asm-400g-0014-data6",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c0e900000015",SYMLINK+="asm-400g-0015-data7",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c12e00000016",SYMLINK+="asm-400g-0016-data8",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de893a0000000da",SYMLINK+="asm-5g-00da-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de8941e000000db",SYMLINK+="asm-5g-00db-grid2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de894a4000000dc",SYMLINK+="asm-5g-00dc-grid3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de80f1f00000039",SYMLINK+="asm-100g-0039-arch1",OWNER="grid",GROUP="asmadmin",MODE="0660"

这里看到这个/lib/udev/scsi_id -g -u -d /dev/sd*查到的结果和存储多路径查到的Lun WWN,除了scsi_id查询的结果首位多一个3,后面是完全一样的。
此时可以使用udevadm进行应用规则:

udevadm control --reload
udevadm trigger

然后查看结果:

[root@xxdb01 ~]# ls -l /dev/asm*
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-100g-0039-arch1 -> sdm
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-000f-data1 -> sdb
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0010-data2 -> sdc
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0011-data3 -> sdd
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0012-data4 -> sde
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0013-data5 -> sdf
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0014-data6 -> sdg
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0015-data7 -> sdh
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-400g-0016-data8 -> sdi
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-5g-00da-grid1 -> sdj
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-5g-00db-grid2 -> sdk
lrwxrwxrwx. 1 root root 3 Sep  8 10:19 /dev/asm-5g-00dc-grid3 -> sdl
[root@xxdb01 ~]# ls -l /dev/sd*
brw-rw----. 1 root disk     8,   0 Sep  8 10:10 /dev/sda
brw-rw----. 1 root disk     8,   1 Sep  8 10:10 /dev/sda1
brw-rw----. 1 root disk     8,   2 Sep  8 10:10 /dev/sda2
brw-rw----. 1 grid asmadmin 8,  16 Sep  8 16:02 /dev/sdb
brw-rw----. 1 grid asmadmin 8,  32 Sep  8 10:19 /dev/sdc
brw-rw----. 1 grid asmadmin 8,  48 Sep  8 10:19 /dev/sdd
brw-rw----. 1 grid asmadmin 8,  64 Sep  8 10:19 /dev/sde
brw-rw----. 1 grid asmadmin 8,  80 Sep  8 10:19 /dev/sdf
brw-rw----. 1 grid asmadmin 8,  96 Sep  8 10:19 /dev/sdg
brw-rw----. 1 grid asmadmin 8, 112 Sep  8 10:19 /dev/sdh
brw-rw----. 1 grid asmadmin 8, 128 Sep  8 10:19 /dev/sdi
brw-rw----. 1 grid asmadmin 8, 144 Sep  8 16:02 /dev/sdj
brw-rw----. 1 grid asmadmin 8, 160 Sep  8 16:02 /dev/sdk
brw-rw----. 1 grid asmadmin 8, 176 Sep  8 16:02 /dev/sdl
brw-rw----. 1 grid asmadmin 8, 192 Sep  8 16:02 /dev/sdm

最终使用asmca创建磁盘组,最终结果为:

[grid@xxdb01 ~]$ asmcmd lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  4194304    102400   102276                0          102276              0             N  ARCH/
MOUNTED  EXTERN  N         512   4096  4194304   3276800  3276620                0         3276620              0             N  DATA/
MOUNTED  NORMAL  N         512   4096  4194304     15360    14320             5120            4600              0             Y  GRID/
[grid@xxdb01 ~]$

可以看到,磁盘组的名字和上面定义的磁盘别名,可以很好的进行管理维护,个人认为这个客户的规范很值得借鉴。