转：谈谈Oracle ASM的限制
今天在客户现场安装Oracle rac时，创建ASM DISKGROUP时出现问题，以下为解决过程。记录之，以备忘。
客户操作系统版本为AIX 6105，Oracle crs版本为10.2.0.5,rdbms和asm版本为10.2.0.1。
当用dbca工具建立磁盘组时，检索不出备选磁盘。在确定rdisk权限等没问题之后，在启动ASM实例之后，尝试在1号节点手动建立磁盘组，不料建磁盘组时出现以下错误。
红色标注可以看到，Oracle识别不了磁盘rhdisk7，但实际上rhdisk7确实存在。由于时间紧张，上google搜索，在一份otn的帖子中，一老外同行也碰到过类似问题，后来在解决问题描述时指出：
询问客户之后，发现本案例中也是raid5 而且rdisk也超过了2T，于是按照网友指出的方法进行创磁盘组尝试。幸好创建成功！
创建成功之后，碰到了ASM重启之后，识别不了磁盘组错误：
或者
解决办法为：
在asm参数文件中显示指明该asm需要mount的磁盘组
为什么ASM不能创建超过2T的硬盘呢？上metalink一查，Oracle针对这一现象，解释为bug：Bug 6453944 - ORA-15196 with ASM disks larger than 2TB [ID 6453944.8]
此外metalink 736891.1对此bug引起的后果有较为详细的描述，其中最直接的后果就是损坏ASM元数据（metadata）：
当然这一bug已经在以下版本修复：
那么除此之外，ASM还有什么限制呢?(metalink:ID 370921.1).我想除了2TB限制，其他限制达到的可能性还是比较低的。
客户操作系统版本为AIX 6105，Oracle crs版本为10.2.0.5,rdbms和asm版本为10.2.0.1。
当用dbca工具建立磁盘组时，检索不出备选磁盘。在确定rdisk权限等没问题之后，在启动ASM实例之后，尝试在1号节点手动建立磁盘组，不料建磁盘组时出现以下错误。
引用
$ sqlplus "/as sysdba"
SQL*Plus: Release 10.2.0.1.0 - Production on Thu Mar 31 13:44:37 2011
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> create diskgroup data1 external redundancy disk '/dev/rhdisk7' size 2047G;
create diskgroup data1 external redundancy disk '/dev/rhdisk7' size 2047G
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error:
SQL*Plus: Release 10.2.0.1.0 - Production on Thu Mar 31 13:44:37 2011
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> create diskgroup data1 external redundancy disk '/dev/rhdisk7' size 2047G;
create diskgroup data1 external redundancy disk '/dev/rhdisk7' size 2047G
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error: 6: No such device or address
Additional information: -1
Additional information: 655360
IBM AIX RISC System/6000 Error:
红色标注可以看到，Oracle识别不了磁盘rhdisk7，但实际上rhdisk7确实存在。由于时间紧张，上google搜索，在一份otn的帖子中，一老外同行也碰到过类似问题，后来在解决问题描述时指出：
引用
Hi All,
I have resolved the issue now.
Since we are using SAN disks with raid 5. The disks size visible to ASm was not correct one.
So what i did is that I added SIZE clause during asm diskgroup creation.
create diskgroup oradata external redundany disk '/dev/rhdisk4' size 1000G;
by issuing this statement now I can create diskgroups.
Remember max size of a SAN disk for ASM is 2TB. We can't go beyond 2TB
I have resolved the issue now.
Since we are using SAN disks with raid 5. The disks size visible to ASm was not correct one.
So what i did is that I added SIZE clause during asm diskgroup creation.
create diskgroup oradata external redundany disk '/dev/rhdisk4' size 1000G;
by issuing this statement now I can create diskgroups.
Remember max size of a SAN disk for ASM is 2TB. We can't go beyond 2TB
询问客户之后，发现本案例中也是raid5 而且rdisk也超过了2T，于是按照网友指出的方法进行创磁盘组尝试。幸好创建成功！
引用
SQL> create diskgroup data external redundancy disk '/dev/rhdisk7' size 1990G;
Diskgroup created.
Diskgroup created.
创建成功之后，碰到了ASM重启之后，识别不了磁盘组错误：
引用
SQL> startup
ASM instance started
Total System Global Area 130023424 bytes
Fixed Size 2019136 bytes
Variable Size 102838464 bytes
ASM Cache 25165824 bytes
ORA-15110: no diskgroups mounted
ASM instance started
Total System Global Area 130023424 bytes
Fixed Size 2019136 bytes
Variable Size 102838464 bytes
ASM Cache 25165824 bytes
ORA-15110: no diskgroups mounted
或者
引用
SQL> startup
ASM instance started
Total System Global Area 130023424 bytes
Fixed Size 2019136 bytes
Variable Size 102838464 bytes
ASM Cache 25165824 bytes
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup
"DATA,DATA1"
ASM instance started
Total System Global Area 130023424 bytes
Fixed Size 2019136 bytes
Variable Size 102838464 bytes
ASM Cache 25165824 bytes
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup
"DATA,DATA1"
解决办法为：
在asm参数文件中显示指明该asm需要mount的磁盘组
引用
asm_diskgroups='DATA','ARCH'
ASM_DISKSTRING ='/dev/rhdisk7','/dev/rhdisk8'
ASM_DISKSTRING ='/dev/rhdisk7','/dev/rhdisk8'
为什么ASM不能创建超过2T的硬盘呢？上metalink一查，Oracle针对这一现象，解释为bug：Bug 6453944 - ORA-15196 with ASM disks larger than 2TB [ID 6453944.8]
引用
ORA-15196 (ASM block corruption) can occur, if LUNs larger than 2TB are
presented to an ASM diskgroup.
As a result of the fix, ORA-15099 will be raised if larger than 2TB size
disk is specified. This is irrespective of the presence of asmlib.
In the future, this limitation will be removed.
Workaround:
Do not add more than 2 TB size disk to a diskgroup.
presented to an ASM diskgroup.
As a result of the fix, ORA-15099 will be raised if larger than 2TB size
disk is specified. This is irrespective of the presence of asmlib.
In the future, this limitation will be removed.
Workaround:
Do not add more than 2 TB size disk to a diskgroup.
此外metalink 736891.1对此bug引起的后果有较为详细的描述，其中最直接的后果就是损坏ASM元数据（metadata）：
引用
There are no clear indications of what can go wrong with ASM LUNs larger than 2TB – but one example is wrapping: when Oracle gets to the end of 2TB, it may wrap back to the start of the LUN and overwrite the ASM metadata
，并导致如下错误发生： 引用
ORA-15196 : invalid ASM block header [kfc.c:7997] [hard_kfbh] [4] [4] [34 != 130]
当然这一bug已经在以下版本修复：
引用
11.2
10.2.0.4 (Server Patch Set)
11.1.0.7 (Server Patch Set)
10.2.0.4 (Server Patch Set)
11.1.0.7 (Server Patch Set)
那么除此之外，ASM还有什么限制呢?(metalink:ID 370921.1).我想除了2TB限制，其他限制达到的可能性还是比较低的。
引用
63 disk groups in a storage system
10,000 ASM disks in a storage system
2 terabyte maximum storage for each ASM disk (the Bug 6453944 allowed larger sizes, but that led to problems, see Note 736891.1 "ORA-15196 WITH ASM DISKS LARGER THAN 2TB")
40 exabyte maximum storage for each storage system
1 million files for each disk group
2.4 terabyte maximum storage for each file
10,000 ASM disks in a storage system
2 terabyte maximum storage for each ASM disk (the Bug 6453944 allowed larger sizes, but that led to problems, see Note 736891.1 "ORA-15196 WITH ASM DISKS LARGER THAN 2TB")
40 exabyte maximum storage for each storage system
1 million files for each disk group
2.4 terabyte maximum storage for each file
=============================
浙公网安备 33010602011771号