vmware server 安装oracle 10G RAC 实践（二）

（注：由于在测试中发现vmware server 2.0创建虚拟机之后ping不通网关主机，查了半天未果，于是换回vmware server 1.0，创建和安装步骤类似，下面还是采用2.0的图）

在了解ORACLE集群原理后开始动手在VMWARE上安装oracle 10G rac,先搜集了一下资料。看到很多网友提到Vmware workstation不支持共享存储，也有人说6.5版本以上可以支持，我的机器上有原先安装的Vmware workstation 5.5,看来是不支持了，于是先下了vmvare server 2.0，它可以在以下网址下到

http://downloads.vmware.com/d/info/datacenter_downloads/vmware_server/2_0

通过申请直接得到注册码。

Oracle clusterware软件下载地址：

http://www.oracle.com/technology/global/cn/software/products/database/clusterware/index.html

oracle database 10Gr2下载地址：

http://www.oracle.com/technology/software/products/database/index.html

一、安装VMWARE SERVER

安装过程就不多说了，直接按照提示来，装完重启电脑。

通过https://myth:8333/ui/登录到虚拟服务器管理界面，

下载安装vmware-vmrc-win32-x86.exe组件包，用来显示console

二、下载RED HAT 4

电驴资源里搜一下，我下的是

[红帽企业级Linux.AS版].TLF-SOFT-Redhat.Enterprise.Linux.AS.V4.0.UPDATE.7.DVD-HOTiSO.iso

Uname显示linux内核版本为

Linux NOD1 2.6.9-78.ELsmp #1 SMP Wed Jul 9 15:39:47 EDT 2008 i686 i686 i386 GNU/Linux

三、新建一个虚拟机（NOD1），安装RED HAT

新建一个linux虚拟机，磁盘大小为10G，磁盘类型为SCSI，网卡采用桥接方式，使用700M内存。

创建共享磁盘

由于我打算采用OCFS2+ASM来存储，Voting Disk和ocr存储在OCFS2上，数据文件和恢复文件存放在ASM上，所以需要再创建4个磁盘文件。

Voting disk至少需要20M，OCR至少需要100M，数据文件和恢复文件大概在不小于2G.

另外再添加一块网卡，选择host-only

创建好后如下所示

安装RED HAT4 update 7

选择手动分区，我的分区如下

软件包选择开发包，X窗口其他的默认，后期安装oracle的时候需要验证依赖的包，缺少再装上去就行了，顺便也可以熟悉一下

安装完之后重新引导，稍微做下配置就可以了。

 

进入系统。接下来是配置集群环境，有一大堆命令

找到nod1目录下的.vmx文件，添加

 

config.version = "8"
virtualHW.version = "4"
scsi0.present = "TRUE"
scsi0.virtualDev = "lsilogic"
memsize = "700"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "localdisk.vmdk"
ide1:0.present = "TRUE"
ide1:0.fileName = "G:\linux\[红帽企业级Linux.AS版].TLF-SOFT-Redhat.Enterprise.Linux.AS.V4.0.UPDATE.7.DVD-HOTiSO.iso"
ide1:0.deviceType = "cdrom-image"
floppy0.present = "FALSE"
Ethernet0.present = "TRUE"
displayName = "NOD1"
guestOS = "rhel4"
priority.grabbed = "normal"
priority.ungrabbed = "normal"

scsi1.present = "TRUE"
scsi1:1.present = "TRUE"
scsi1:1.fileName = "D:\linux\sharestoredisk\ocfs2disk.vmdk"
scsi1:1.mode = "independent-persistent"
scsi1.virtualDev = "lsilogic"
ide1:0.autodetect = "TRUE"

scsi1:2.present = "TRUE"
scsi1:2.fileName = "D:\linux\sharestoredisk\asmdisk1.vmdk"
scsi1:2.mode = "independent-persistent"

scsi1:3.present = "TRUE"
scsi1:3.fileName = "D:\linux\sharestoredisk\asmdisk2.vmdk"
scsi1:3.mode = "independent-persistent"

scsi1:4.present = "TRUE"
scsi1:4.fileName = "D:\linux\sharestoredisk\asmdisk3.vmdk"
scsi1:4.mode = "independent-persistent"

Ethernet1.present = "TRUE"
Ethernet1.connectionType = "hostonly"

disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
scsi1.sharedBus = "virtual"


scsi1:1.deviceType = "disk"
scsi1:2.deviceType = "disk"
scsi1:3.deviceType = "disk"
scsi1:4.deviceType = "disk"

scsi0:0.redo = ""
scsi1:1.redo = ""
scsi1:2.redo = ""
scsi1:3.redo = ""
scsi1:4.redo = ""
ide1:0.startConnected = "TRUE"
ethernet0.addressType = "generated"
ethernet1.addressType = "generated"
uuid.location = "56 4d 1d b2 5d 55 d9 56-e8 c5 bd 75 7e d7 0f e0"
uuid.bios = "56 4d 1d b2 5d 55 d9 56-e8 c5 bd 75 7e d7 0f e0"
ethernet0.generatedAddress = "00:0c:29:d7:0f:e0"
ethernet0.generatedAddressOffset = "0"
ethernet1.generatedAddress = "00:0c:29:d7:0f:ea"
ethernet1.generatedAddressOffset = "10"

workingDir = "."

 

创建组oinstall,dba,用户oracle

groupadd oinstall

groupadd dba

useradd -d /home/oracle -g oinstall -G dba oracle

passwd oracle

Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.

添加环境变量

[root@NOD1 /]# vi /home/oracle/.bash_profile

export TMP=/tmp

export TMPDIR=$TMP

export ORACLE_BASE=/home/ora10g

export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1

export ORACLE_SID=orcl

export ORACLE_TERM=xterm

export PATH=/usr/sbin:$PATH

export PATH=$ORACLE_HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin/

export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

ulimit -u 16384 -n 65536

umask 022

~

创建ORACLE_BASE目录，注意文件夹权限

mkdir /home/ora10g

chown oracle.oinstall /home/ora10g

设置内核参数

[root@NOD1 /]# vi /etc/sysctl.conf

kernel.shmall = 2097152

kernel.shmmax = 536870912

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

fs.file-max = 65536

net.ipv4.ip_local_port_range = 1024 65000

net.core.rmem_default = 1048576

net.core.rmem_max = 1048576

net.core.wmem_default = 262144

net.core.wmem_max = 262144

让设置生效。

[root@node1 Server]# sysctl -p

提高 Oracle 用户的 shell 限制
设置oracle使用的文件数权限

[root@NOD1 /]# vi /etc/security/limits.conf

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

 

修改安全限制

[root@NOD1 /]# vi /etc/pam.d/login

session    required     /lib/security/pam_limits.so

配置Hangcheck计时器

[root@NOD1 /]# vi /etc/rc.local

modprobe hangcheck-timer hangcheck-tick=30 hangcheck_margin=180

修改/etc/hosts文件

128.1.100.144 nod1
128.1.100.143 nod2

128.1.100.201 nod1-vip
128.1.100.202 nod2-vip

10.10.10.100 nod1-priv
10.10.10.101 nod2-priv

磁盘分区

[root@NOD1 /]# ./sbin/fdisk -l

 

Disk /dev/sda: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

/dev/sda1   *           1          63      506016   83 Linux

/dev/sda2              64         445     3068415   83 Linux

/dev/sda3             446         611     1333395   82 Linux swap

/dev/sda4             612        1305     5574555    5 Extended

/dev/sda5             612        1305     5574523+ 83 Linux

 

Disk /dev/sdb: 858 MB, 858993152 bytes

64 heads, 32 sectors/track, 819 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

 

   Device Boot      Start         End      Blocks   Id System

 

Disk /dev/sdc: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

 

Disk /dev/sdd: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

 

Disk /dev/sde: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

[root@NOD1 /]# ./sbin/fdisk /dev/sdb

 

Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p\

Partition number (1-4): 1

First cylinder (1-819, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-819, default 819):   

Using default value 819

 

Command (m for help): w

The partition table has been altered!

 

Calling ioctl() to re-read partition table.

Syncing disks.

[root@NOD1 /]#

依次对sdc,sde,sdf进行分区，分区完之后查看如下

[root@NOD1 /]# ./sbin/fdisk -l

 

Disk /dev/sda: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

/dev/sda1   *           1          63      506016   83 Linux

/dev/sda2              64         445     3068415  83 Linux

/dev/sda3             446         611     1333395   82 Linux swap

/dev/sda4             612        1305     5574555    5 Extended

/dev/sda5             612        1305     5574523+ 83 Linux

 

Disk /dev/sdb: 858 MB, 858993152 bytes

64 heads, 32 sectors/track, 819 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes

 

   Device Boot      Start         End      Blocks   Id System

/dev/sdb1               1         819      838640   83 Linux

 

Disk /dev/sdc: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

/dev/sdc1               1         391     3140676   83 Linux

 

Disk /dev/sdd: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

/dev/sdd1               1         391     3140676   83 Linux

 

Disk /dev/sde: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks   Id System

/dev/sde1        

 

下载ASMLIB包，可以通过以下链接下载

http://www.oracle.com/technology/software/tech/linux/asmlib/rhel4.html

-2.6.9-78的我全下了……….

oracleasm-2.6.9-78.ELxenU-2.0.5-1.el4.i686.rpm

oracleasm-2.6.9-78.ELsmp-2.0.5-1.el4.i686.rpm

oracleasm-2.6.9-78.ELhugemem-2.0.5-1.el4.i686.rpm

oracleasm-2.6.9-78.EL-debuginfo-2.0.5-1.el4.i686.rpm

oracleasm-2.6.9-78.EL-2.0.5-1.el4.i686.rpm

oracleasm-support-2.1.3-1.el4.i386.rpm

oracleasmlib-2.0.4-1.el4.i386.rpm

 

在安装的时候提示缺少kernel-xenU包和kernel-hugemem包，在安装盘里找到rpm上

检查asm包

[root@NOD1 home]# rpm -qa | grep oracleasm

oracleasm-2.6.9-78.EL-debuginfo-2.0.5-1.el4

oracleasm-2.6.9-78.EL-2.0.5-1.el4

oracleasm-2.6.9-78.ELxenU-2.0.5-1.el4

oracleasmlib-2.0.4-1.el4

oracleasm-2.6.9-78.ELhugemem-2.0.5-1.el4

oracleasm-2.6.9-78.ELsmp-2.0.5-1.el4

oracleasm-support-2.1.3-1.el4

[root@NOD1 home]#

 

 

映射ASM磁盘，添加

 

[root@NOD1 ~]# vi /etc/sysconfig/rawdevices

/dev/raw/raw1 /dev/sdc

/dev/raw/raw2 /dev/sdd

/dev/raw/raw3 /dev/sde

 

立即 生效，授权

[root@NOD1 ~]# /sbin/service rawdevices restart

Assigning devices:

           /dev/raw/raw1 -->   /dev/sdc

/dev/raw/raw1: bound to major 8, minor 32

           /dev/raw/raw2 -->   /dev/sdd

/dev/raw/raw2: bound to major 8, minor 48

           /dev/raw/raw3 -->   /dev/sde

/dev/raw/raw3: bound to major 8, minor 64

done

[root@NOD1 ~]# chown oracle:dba /dev/raw/raw[1-3]

[root@NOD1 ~]# chmod 660 /dev/raw/raw[1-3]

[root@NOD1 ~]#

 

切换到ORACLE用户，建立ASM磁盘快捷方式

[oracle@NOD1 ~]$ mkdir $ORACLE_BASE/oradata

[oracle@NOD1 ~]$ mkdir $ORACLE_BASE/admin

[oracle@NOD1 ~]$ mkdir $ORACLE_BASE/oradata/orcl

[oracle@NOD1 ~]$ ln -sf /dev/raw/raw1 /home/ora10g/oradata/orcl/asmdisk1

[oracle@NOD1 ~]$ ln -sf /dev/raw/raw2 /home/ora10g/oradata/orcl/asmdisk2

[oracle@NOD1 ~]$ ln -sf /dev/raw/raw3 /home/ora10g/oradata/orcl/asmdisk3

[oracle@NOD1 ~]$

 

修改 /etc/udev/permissions.d/50-udev.permissions，使ORACLE用户作为引导原始设备的拥有者

[root@NOD1 ~]# vi /etc/udev/permissions.d/50-udev.permissions

注销 raw/*:root:disk:0660

添加raw/*:oracle:dba:0660