使用asmfd绑定磁盘
1.停止crs，配置asmfd
[root@lxtrac04 bin]# ./asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.
[root@lxtrac04 bin]#
2.检查asmfd的状态
[root@lxtrac04 bin]# ./asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'lxtrac04'
[root@lxtrac04 bin]#
3.使用独占模式nocrs启动集群
[root@lxtrac04 bin]# ./crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'lxtrac04'
CRS-2672: Attempting to start 'ora.evmd' on 'lxtrac04'
CRS-2672: Attempting to start 'ora.mdnsd' on 'lxtrac04'
CRS-2676: Start of 'ora.cssdmonitor' on 'lxtrac04' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'lxtrac04' succeeded
CRS-2676: Start of 'ora.evmd' on 'lxtrac04' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'lxtrac04'
CRS-2676: Start of 'ora.gpnpd' on 'lxtrac04' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'lxtrac04'
CRS-2676: Start of 'ora.gipcd' on 'lxtrac04' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'lxtrac04'
CRS-2672: Attempting to start 'ora.diskmon' on 'lxtrac04'
CRS-2676: Start of 'ora.diskmon' on 'lxtrac04' succeeded
CRS-2676: Start of 'ora.cssd' on 'lxtrac04' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'lxtrac04'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'lxtrac04'
CRS-2672: Attempting to start 'ora.ctssd' on 'lxtrac04'
CRS-2676: Start of 'ora.drivers.acfs' on 'lxtrac04' succeeded
CRS-2676: Start of 'ora.ctssd' on 'lxtrac04' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'lxtrac04' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'lxtrac04'
CRS-2676: Start of 'ora.asm' on 'lxtrac04' succeeded
[root@lxtrac04 bin]#
4.查看集群的asm_diskstring
[root@lxtrac04 bin]# ./asmcmd dsget
parameter: AFD:*
profile:AFD:*
[root@lxtrac04 bin]#
5.设置新的集群asm_diskstring
[root@lxtrac04 bin]# ./asmcmd dsset '/dev/sd*','AFD:*'
[root@lxtrac04 bin]# ./asmcmd dsget
parameter:/dev/sd*, AFD:*
profile:/dev/sd*,AFD:*
[root@lxtrac04 bin]#
6.对磁盘设置标签，需要加 --migrate
[root@lxtrac04 bin]# ./asmcmd afd_label DATA01 /dev/sdd5
disk /dev/sdd5 is already provisioned for ASM
ASMCMD-9513: ASM disk label set operation failed.
[root@lxtrac04 bin]# ./asmcmd afd_label DATA01 /dev/sdd5 --migrate
[root@lxtrac04 bin]#
……………………
7.关闭crs
[root@lxtrac04 bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'lxtrac04'
CRS-2673: Attempting to stop 'ora.ctssd' on 'lxtrac04'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'lxtrac04'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'lxtrac04'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'lxtrac04'
CRS-2677: Stop of 'ora.drivers.acfs' on 'lxtrac04' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'lxtrac04' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'lxtrac04'
CRS-2673: Attempting to stop 'ora.asm' on 'lxtrac04'
CRS-2677: Stop of 'ora.mdnsd' on 'lxtrac04' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'lxtrac04' succeeded
CRS-2677: Stop of 'ora.evmd' on 'lxtrac04' succeeded
CRS-2677: Stop of 'ora.asm' on 'lxtrac04' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'lxtrac04'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'lxtrac04' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'lxtrac04'
CRS-2677: Stop of 'ora.cssd' on 'lxtrac04' succeeded
CRS-2673: Attempting to stop 'ora.driver.afd' on 'lxtrac04'
CRS-2673: Attempting to stop 'ora.gipcd' on 'lxtrac04'
CRS-2677: Stop of 'ora.driver.afd' on 'lxtrac04' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'lxtrac04' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'lxtrac04' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@lxtrac04 bin]#
8.取消udev的使用
[root@lxtrac04 bin]# cat /etc/udev/rules.d/99-oracle-asm.rules
#KERNEL=="sdd[1-9]",ACTION=="add",OWNER="grid", GROUP="asmadmin", MODE="0660"
#KERNEL=="sde[1-9]",ACTION=="add",OWNER="grid", GROUP="asmadmin", MODE="0660"
[root@lxtrac04 bin]# systemctl restart systemd-udevd.service
[root@lxtrac04 bin]# udevadm test /sys/class/block/sdd1 --查看udev生效
[root@lxtrac04 bin]# ll /dev/sd* udev生效，磁盘都是root:disk。grid对磁盘没有读取权限。
brw-rw---- 1 root disk 8, 0 Mar 19 15:19 /dev/sda
brw-rw---- 1 root disk 8, 1 Mar 19 15:19 /dev/sda1
brw-rw---- 1 root disk 8, 2 Mar 19 15:19 /dev/sda2
brw-rw---- 1 root disk 8, 16 Mar 19 15:19 /dev/sdb
brw-rw---- 1 root disk 8, 32 Mar 19 15:19 /dev/sdc
brw-rw---- 1 root disk 8, 48 Mar 19 15:19 /dev/sdd
brw-rw---- 1 root disk 8, 49 Mar 19 15:35 /dev/sdd1
brw-rw---- 1 root disk 8, 50 Mar 19 15:35 /dev/sdd2
brw-rw---- 1 root disk 8, 51 Mar 19 15:35 /dev/sdd3
brw-rw---- 1 root disk 8, 52 Mar 19 15:35 /dev/sdd4
brw-rw---- 1 root disk 8, 53 Mar 19 15:35 /dev/sdd5
brw-rw---- 1 root disk 8, 54 Mar 19 15:35 /dev/sdd6
brw-rw---- 1 root disk 8, 64 Mar 19 15:19 /dev/sde
brw-rw---- 1 root disk 8, 65 Mar 19 15:36 /dev/sde1
brw-rw---- 1 root disk 8, 66 Mar 19 15:36 /dev/sde2
brw-rw---- 1 root disk 8, 67 Mar 19 15:36 /dev/sde3
[root@lxtrac04 bin]#
9.启动crs。启动成功
[root@lxtrac04 bin]# ./crsctl start crs
[root@lxtrac04 bin]# ./crsctl stat res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE lxtrac04 STABLE
ora.cluster_interconnect.haip
1 ONLINE ONLINE lxtrac04 STABLE
ora.crf
1 ONLINE ONLINE lxtrac04 STABLE
ora.crsd
1 ONLINE ONLINE lxtrac04 STABLE
ora.cssd
1 ONLINE ONLINE lxtrac04 STABLE
ora.cssdmonitor
1 ONLINE ONLINE lxtrac04 STABLE
ora.ctssd
1 ONLINE ONLINE lxtrac04 OBSERVER,STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.driver.afd
1 ONLINE ONLINE lxtrac04 STABLE
ora.drivers.acfs
1 ONLINE ONLINE lxtrac04 STABLE
ora.evmd
1 ONLINE ONLINE lxtrac04 STABLE
ora.gipcd
1 ONLINE ONLINE lxtrac04 STABLE
ora.gpnpd
1 ONLINE ONLINE lxtrac04 STABLE
ora.mdnsd
1 ONLINE ONLINE lxtrac04 STABLE
ora.storage
1 ONLINE ONLINE lxtrac04 STABLE
--------------------------------------------------------------------------------
[root@lxtrac04 bin]#
10.检查asmfd的配置
[root@lxtrac04 bin]# ./asmcmd
ASMCMD> afd_lsdsk
--------------------------------------------------------------------------------
Label Filtering Path
================================================================================
DATA01 ENABLED /dev/sdd5
NEWOCR01 ENABLED /dev/sde1
NEWOCR02 ENABLED /dev/sde2
NEWOCR03 ENABLED /dev/sde3
OCR01 ENABLED /dev/sdd1
OCR02 ENABLED /dev/sdd2
OCR03 ENABLED /dev/sdd3
TEST01 ENABLED /dev/sdd6
ASMCMD>
在sqlplus中查询，已使用的asmfd
SQL> select NAME,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS ,PATH from v$asm_disk;
NAME MOUNT_S HEADER_STATU MODE_ST PATH
------------------------- ------- ------------ ------- -------------------------------------------------------
CLOSED PROVISIONED ONLINE AFD:TEST01
DATA01 CACHED MEMBER ONLINE AFD:DATA01
NEWOCR01 CACHED MEMBER ONLINE AFD:NEWOCR01
NEWOCR02 CACHED MEMBER ONLINE AFD:NEWOCR02
NEWOCR03 CACHED MEMBER ONLINE AFD:NEWOCR03
OCR01 CACHED MEMBER ONLINE AFD:OCR01
OCR02 CACHED MEMBER ONLINE AFD:OCR02
OCR03 CACHED MEMBER ONLINE AFD:OCR03
8 rows selected.
SQL>
参考资料：http://www.oracle.com/technetwork/cn/articles/database/asmfd-2398572-zhs.html
