重装操作系统后起集群

文档控制		项目名称
		文档名称	测试报告

1.文档属性
文档编号：
2.修改记录
日期	修改人	修改记录		备注
2017-06-23	hubert	V1.0


3.审阅记录
日期	审阅人	公司名称及相关人员职务		备注



4.分发记录
日期	分发人	公司名称及相关人员职务		备注

引言

2.1 编写目的

本测试报告对Oracle数据库硬件服务器升级操作系统后数据库集群能否启用验证测试报告。

2.2 方案背景

根据要求，操作系统需要从Linux5.8升级到Linux6.9后，重启数据库集群。根据实际情况，我方对Oracle数据库集群重新启用报告如下。

方案设计

3.1 升级方案

操作系统重装，由甲方完成。我方负责起数据库集群，并使数据库集群正常运行，不影响业务。

3.2 回退方案

我方负责数据库备份，发生不可预估后果，及时回退。

测试概况

4.1 测试配置

节点1

主机名	ol112301
主机OS	Linux red hat 5.8 x86_64
主机IP	192.168.1.121 192.168.1.123
数据库版本	Oracle 11.2.0.3.0

节点2

主机名	ol112302
主机OS	Linux red hat 5.8 x86_64
主机IP	192.168.1.122 192.168.1.124
数据库版本	Oracle 11.2.0.3.0

4.2 测试范围

本次测试为实际操作测试，验证方案的可行性。包括

操作系统升级Linux5.8升级到Linux6.9
重启Oracle-RAC集群

4.3 测试人员

本次测试时间为2016年10月6日，参与测试人员：wanganshen

测试准备

5.1检查数据库集群状况

crsctl status res -t

Asm实例检查：

[grid@shareraca ~]$ asmcmd -p

ASMCMD [+] > lsdg

State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name

MOUNTED EXTERN N 512 4096 1048576 2048000 972492 0 972492 0 N GBK_R5DG/

MOUNTED NORMAL N 512 4096 1048576 3072 2146 1024 561 0 Y OCRDG/

MOUNTED EXTERN N 512 4096 4194304 1024000 732540 0 732540 0 N ORADATA/

MOUNTED EXTERN N 512 4096 4194304 1024000 1023248 0 1023248 0 N ORAFLASH/

MOUNTED EXTERN N 512 4096 1048576 6144000 553090 0 553090 0 N UTF_R5DG/

MOUNTED EXTERN N 512 4096 1048576 3072000 3058022 0 3058022 0 N YPGTDG/

[root@shareraca ~]# crsctl query css votedisk

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 595e699d76d94f6abf060082b4c8da05 (ORCL:VSPG_0086) [OCRDG]

2. ONLINE 093f87ee413d4f6fbfaa876c4c9cca43 (ORCL:VSPG_0087) [OCRDG]

3. ONLINE 759dd37ae2004f22bf3f6046dc801721 (ORCL:VSPG_0088) [OCRDG]

备份OCR文件

ocrconfig -export /oracle/upos/ocr.bak

检查是否使用oracle asmlib

[root@shareraca /]# /etc/init.d/oracleasm listdisks

USPVM1_002B

VSPG_0086

VSPG_0087

VSPG_0088

VSPG_0112

VSPG_0113

VSPG_0114

VSPG_020A

VSPG_020B

VSPG_020C

VSPG_020D

VSPG_020E

VSPG_020F

VSPG_0210

VSPG_0228

VSPG_0229

VSPG_022A

VSPG_0232

VSPG_0238

VSPG_0239

[root@shareraca disks]# oracleasm querydisk -v -p USPVM1_002B

Disk "USPVM1_002B" is a valid ASM disk

/dev/mapper/USPVM002b: LABEL="USPVM1_002B" TYPE="oracleasm"

/dev/sdk: LABEL="USPVM1_002B" TYPE="oracleasm"

/dev/sdq: LABEL="USPVM1_002B" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0086

Disk "VSPG_0086" is a valid ASM disk

/dev/mapper/mpath14: LABEL="VSPG_0086" TYPE="oracleasm"

/dev/sdab: LABEL="VSPG_0086" TYPE="oracleasm"

/dev/sdae: LABEL="VSPG_0086" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0087

Disk "VSPG_0087" is a valid ASM disk

/dev/mapper/mpath15: LABEL="VSPG_0087" TYPE="oracleasm"

/dev/sdac: LABEL="VSPG_0087" TYPE="oracleasm"

/dev/sdaf: LABEL="VSPG_0087" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0088

Disk "VSPG_0088" is a valid ASM disk

/dev/mapper/mpath16: LABEL="VSPG_0088" TYPE="oracleasm"

/dev/sdad: LABEL="VSPG_0088" TYPE="oracleasm"

/dev/sdag: LABEL="VSPG_0088" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0112

Disk "VSPG_0112" is a valid ASM disk

/dev/mapper/mpath17: LABEL="VSPG_0112" TYPE="oracleasm"

/dev/sdah: LABEL="VSPG_0112" TYPE="oracleasm"

/dev/sdak: LABEL="VSPG_0112" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0113

Disk "VSPG_0113" is a valid ASM disk

/dev/mapper/mpath18: LABEL="VSPG_0113" TYPE="oracleasm"

/dev/sdai: LABEL="VSPG_0113" TYPE="oracleasm"

/dev/sdal: LABEL="VSPG_0113" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0114

Disk "VSPG_0114" is a valid ASM disk

/dev/mapper/mpath19: LABEL="VSPG_0114" TYPE="oracleasm"

/dev/sdaj: LABEL="VSPG_0114" TYPE="oracleasm"

/dev/sdam: LABEL="VSPG_0114" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_020A

Disk "VSPG_020A" is a valid ASM disk

/dev/mapper/mpath21: LABEL="VSPG_020A" TYPE="oracleasm"

/dev/sdao: LABEL="VSPG_020A" TYPE="oracleasm"

/dev/sdas: LABEL="VSPG_020A" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_020B

Disk "VSPG_020B" is a valid ASM disk

/dev/mapper/mpath22: LABEL="VSPG_020B" TYPE="oracleasm"

/dev/sdap: LABEL="VSPG_020B" TYPE="oracleasm"

/dev/sdat: LABEL="VSPG_020B" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_020C

Disk "VSPG_020C" is a valid ASM disk

/dev/mapper/mpath23: LABEL="VSPG_020C" TYPE="oracleasm"

/dev/sdaq: LABEL="VSPG_020C" TYPE="oracleasm"

/dev/sdau: LABEL="VSPG_020C" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_020D

Disk "VSPG_020D" is a valid ASM disk

/dev/mapper/mpath24: LABEL="VSPG_020D" TYPE="oracleasm"

/dev/sdav: LABEL="VSPG_020D" TYPE="oracleasm"

/dev/sday: LABEL="VSPG_020D" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_020E

Disk "VSPG_020E" is a valid ASM disk

/dev/mapper/mpath25: LABEL="VSPG_020E" TYPE="oracleasm"

/dev/sdaw: LABEL="VSPG_020E" TYPE="oracleasm"

/dev/sdaz: LABEL="VSPG_020E" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_020F

Disk "VSPG_020F" is a valid ASM disk

/dev/mapper/mpath26: LABEL="VSPG_020F" TYPE="oracleasm"

/dev/sdax: LABEL="VSPG_020F" TYPE="oracleasm"

/dev/sdba: LABEL="VSPG_020F" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0210

Disk "VSPG_0210" is a valid ASM disk

/dev/mapper/mpath27: LABEL="VSPG_0210" TYPE="oracleasm"

/dev/sdbc: LABEL="VSPG_0210" TYPE="oracleasm"

/dev/sdbb: LABEL="VSPG_0210" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0228

Disk "VSPG_0228" defines a device with no label

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0229

oracleasm querydisk -v -p VSPG_022A

Disk "VSPG_0229" is a valid ASM disk

oracleasm querydisk -v -p VSPG_0232

oracleasm querydisk -v -p VSPG_0238

oracleasm querydisk -v -p VSPG_0239

/dev/mapper/mpath31: LABEL="VSPG_0229" TYPE="oracleasm"

/dev/sdbe: LABEL="VSPG_0229" TYPE="oracleasm"

/dev/sdbh: LABEL="VSPG_0229" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_022A

Disk "VSPG_022A" is a valid ASM disk

/dev/mapper/mpath32: LABEL="VSPG_022A" TYPE="oracleasm"

/dev/sdbi: LABEL="VSPG_022A" TYPE="oracleasm"

/dev/sdbf: LABEL="VSPG_022A" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0232

Disk "VSPG_0232" is a valid ASM disk

/dev/mapper/mpath33: LABEL="VSPG_0232" TYPE="oracleasm"

/dev/sdbj: LABEL="VSPG_0232" TYPE="oracleasm"

/dev/sdbk: LABEL="VSPG_0232" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0238

Disk "VSPG_0238" is a valid ASM disk

/dev/mapper/VSPG_0238: LABEL="VSPG_0238" TYPE="oracleasm"

/dev/sdbn: LABEL="VSPG_0238" TYPE="oracleasm"

/dev/sdbl: LABEL="VSPG_0238" TYPE="oracleasm"

[root@shareraca disks]# oracleasm querydisk -v -p VSPG_0239

Disk "VSPG_0239" is a valid ASM disk

/dev/sdbm: LABEL="VSPG_0239" TYPE="oracleasm"

/dev/mapper/VSPG_0239: LABEL="VSPG_0239" TYPE="oracleasm"

/dev/sdbo: LABEL="VSPG_0239" TYPE="oracleasm

Su – grid

[grid@shareraca ~]$ ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3

Total space (kbytes) : 262120

Used space (kbytes) : 3692

Available space (kbytes) : 258428

ID : 1086104180

Device/File Name : +ocrdg

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check bypassed due to non-privileged user

Sqlplus / as sysasm

SQL> l

1* select GROUP_NUMBER,DISK_NUMBER,STATE,NAME,PATH from v$asm_disk

SQL> /

GROUP_NUMBER DISK_NUMBER STATE NAME PATH

------------ ----------- --------------- -------------------- -------------------------

0 0 NORMAL ORCL:VSPG_0229

0 1 NORMAL ORCL:VSPG_022A

0 3 NORMAL ORCL:VSPG_0239

0 14 NORMAL ORCL:VSPG_0228

3 2 NORMAL USPVM1_002B ORCL:USPVM1_002B

1 3 NORMAL VSPG_0086 ORCL:VSPG_0086

1 4 NORMAL VSPG_0087 ORCL:VSPG_0087

1 5 NORMAL VSPG_0088 ORCL:VSPG_0088

5 0 NORMAL VSPG_0112 ORCL:VSPG_0112

5 1 NORMAL VSPG_0113 ORCL:VSPG_0113

4 1 NORMAL VSPG_0114 ORCL:VSPG_0114

GROUP_NUMBER DISK_NUMBER STATE NAME PATH

------------ ----------- --------------- -------------------- -------------------------

6 0 NORMAL VSPG_020A ORCL:VSPG_020A

6 1 NORMAL VSPG_020B ORCL:VSPG_020B

6 2 NORMAL VSPG_020C ORCL:VSPG_020C

7 0 NORMAL VSPG_020D ORCL:VSPG_020D

7 1 NORMAL VSPG_020E ORCL:VSPG_020E

7 2 NORMAL VSPG_020F ORCL:VSPG_020F

7 4 NORMAL VSPG_0210 ORCL:VSPG_0210

7 5 NORMAL VSPG_0232 ORCL:VSPG_0232

7 7 NORMAL VSPG_0238 ORCL:VSPG_0238

20 rows selected.

备份multipath.conf：

OS 升级需要备份Oracle的目录：

/etc/ora*

[root@shareraca /]# ls -lrt /etc/ora*

-rw-r--r-- 1 root root 60 2013-03-28 /etc/oraInst.loc

-rw-rw-r-- 1 oracle oinstall 1225 2016-11-24 /etc/oratab

-rw-r--r-- 1 oracle oinstall 55 02-14 12:02 /etc/oragchomelist

/etc/oracle:

total 2236

drwxr-xr-x 3 root oinstall 4096 2013-03-28 scls_scr

drwxrwxr-x 5 root oinstall 4096 2013-03-28 oprocd

-rw-r--r-- 1 root root 0 2013-03-28 olr.loc.orig

-rw-r--r-- 1 root oinstall 93 2013-03-28 olr.loc

-rw-r--r-- 1 root root 0 2013-03-28 ocr.loc.orig

-rw-r--r-- 1 root oinstall 38 2013-03-28 ocr.loc

drwxrwx--- 2 root oinstall 4096 2013-03-28 lastgasp

-rws--x--- 1 root oinstall 2261972 2016-10-26 setasmgid

/tmp/

/etc/hosts

/etc/sysctl.conf

/etc/security.limits.conf

/etc/pam.d/login

/etc/multipath.conf

/etc/shadow

/etc/passwd

/home/oracle/.bash_profile

/home/grid/.bash_profile

[root@shareraca /]# id oracle

uid=502(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),504(asmadmin),506(asmdba),503(oper)

[root@shareraca /]# id grid

uid=501(grid) gid=501(oinstall) groups=501(oinstall),504(asmadmin),506(asmdba),507(asmoper)

数据库备份：

Rman备份。

关闭数据库集群：

Crsctl stop cluster –all

等待操作系统重装完毕后：环境变量的设置：

拷贝备份文件至原有目录下：

/etc/hosts

[root@shareraca yum.repos.d]# more /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

10.137.248.157 shareraca

10.137.248.158 shareracb

10.137.248.159 shareraca-vip

10.137.248.160 shareracb-vip

10.137.248.175 sharerac-scan

192.168.248.157 shareraca-priv

192.168.248.158 shareracb-priv

Cp –R oracle /etc/

Cp –R tmp /

Cp –R sysctl.conf /etc/

Cp –R limits.conf /etc/security/

cp –R login /etc/pam.d/

修改 /etc/profile 文件

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then

ulimit -u 16384 -n 65536

umask 022

检查oracle 用户

Id oracle

检查互信关系：

Ssh node2

Ssh node1

检查oracle 用户的环境变量 .bash_profile

export ORACLE_BASE=/oracle/base

export ORACLE_HOME=$ORACLE_BASE/db11.2.0.3

export ORACLE_SID=gbkrac1

export DB_NAME=gbkrac

export CRS_HOME=/oracle/grid/crs11.2.0.3

export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$HOME/bin:$CRS_HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH

export LANG=en_US.utf8

export NLS_LANG=american_america.ZHS16GBK

export EDITOR=vi

export PATH

alias sdba='sqlplus / as sysdba'

检查grid用户

Id oracle

检查互信关系：

Ssh node1

ssh node2

检查grid用户的环境变量 .bash_profile

PATH=$PATH:$HOME/bin

export ORACLE_BASE=/oracle/grid/crs11g_base

export ORACLE_HOME=/oracle/grid/crs11.2.0.3

export ORACLE_SID=+ASM1

export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH

export LANG=en_US.utf8

export NLS_LANG=american_america.AL32UTF8

export EDITOR=vi

export PATH

互信配置的脚本：

./sshUserSetup.sh -user oracle -hosts shareracb shareraca -advanced -exverify -confirm

./sshUserSetup.sh –user grid -hosts shareraca shareracb -advanced -exverify -confirm

（这个脚本在安装GI解压的软件包下就有的脚本，本人认为还是很方便的。）

安装oracle所需要的rpm包

yum源的配置：

本地yum源

[base]

name=redhatbaseurl=file:///mnt

enabled=1

gpgcheck=0

若是内网已经配置好的yum源则可以直接安装rpm包

Yum 源一

[root@shareraca yum.repos.d]# more rhel-debuginfo.repo

[rhel-debuginfo]

name=Red Hat Enterprise Linux $releasever - $basearch - Debug

baseurl=ftp://ftp.redhat.com/pub/redhat/linux/enterprise/$releasever/en/os/$basearch/Debuginfo/

enabled=0

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

yum源二

[root@shareraca yum.repos.d]# more centos.repo

[Cluster]

name=Red Hat Enterprise Linux $releasever-$basearch-Cluster

baseurl=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/Cluster

enabled=1

gpgcheck=1

gpgkey=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/RPM-GPG-KEY-redhat-release

[ClusterStorage]

name=Red Hat Enterprise Linux $releasever-$basearch-ClusterStorage

baseurl=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/ClusterStorage

enabled=1

gpgcheck=1

gpgkey=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/RPM-GPG-KEY-redhat-release

[Server]

name=Red Hat Enterprise Linux $releasever-$basearch-Server

baseurl=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/Server

enabled=1

gpgcheck=1

gpgkey=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/RPM-GPG-KEY-redhat-release

[VT]

name=Red Hat Enterprise Linux $releasever-$basearch-VT

baseurl=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/VT

enabled=1

gpgcheck=1

gpgkey=http://repos.zpepc.com.cn/redhat/$releasever/os/$basearch/RPM-GPG-KEY-redhat-release

(温馨提示：注意使用Linux的tab键补齐)

yum –y install gcc*

yum –y install elf*

yum –y install compat*

yum –y install libaio*

yum –y install libstd*

yum –y install unixODBC*

yum –y install ksh

(根据提示，需要安装的rpm包)

注意（重新安装ASM cuq那个rpm包）

安装ASM lib包

对应原有的ASM 磁盘信息：

开始扫盘：

/etc/init.d/oracleasm listdisks

/etc/init.d/oraclescan disks

扫盘信息如下：

[root@shareraca ~]# /etc/init.d/oracleasm listdisks

USPVM1_002B

VSPG_0086

VSPG_0087

VSPG_0088

VSPG_0112

VSPG_0113

VSPG_0114

VSPG_020A

VSPG_020B

VSPG_020C

VSPG_020D

VSPG_020E

VSPG_020F

VSPG_0210

VSPG_0228

VSPG_0229

VSPG_022A

VSPG_0232

VSPG_0238

VSPG_0239

列出所有的asm磁盘信息，确认并检查信息完全正确后：

重新config 集群信息

脚本在：

打开xmanager

调出图形化集群配置信息：

（说明：UPOS是测试机主机名。）

/etc/hosts 实际环境

[root@shareraca yum.repos.d]# more /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

10.137.248.157 shareraca

10.137.248.158 shareracb

10.137.248.159 shareraca-vip

10.137.248.160 shareracb-vip

10.137.248.175 sharerac-scan

192.168.248.157 shareraca-priv

192.168.248.158 shareracb-priv

用grid身份执行：

cd $ORACLE_HOME/crs/config/

export DISPLAY=ip:0.0

若是config配置不成功，则需要执行rootcrs.pl 脚本。

/u01/app/11.2.0/grid/crs/install/rootcrs.pl -verbose -deconfig -force

关闭seLinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

（测试环境）

xhost +

/oracle/grid/crs11.2.0.3/crs/config/ config.sh

选择configure Oracle grid

要和/etc/hosts scan-ip name保持一致

Chmod -R 777 /tmp (把原有备份的tmp目录拷贝过来)

/tmp/CVU_11.2.0.3.0_oracle、grid中的所有文件缺少执行的权限，后来在两个节点上都使用chmod +x *命令加上执行权限就解决了

解决一下问题后，可忽略next

脚本在所有节点以root身份执行，顺序执行。执行完毕点击OK。

执行完毕，集群和databaase instance 已经起来。

数据库升级部分

5.7 检查无效对象

SQL> set lines 200 pages 999

SQL> Col owner for a30

SQL> Col object_name for a40

SQL> Col object_type for a30

SQL> SELECT owner, object_name, object_type, status

2 FROM dba_objects

3 WHERE status <> 'VALID'

4 ORDER BY owner, object_name;

no rows selected

记录升级前无效对象的情况，升级后便于进行对比

5.8 检查数据文件、控制文件以及参数文件目录

SQL>

SQL> show parameter spfile

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

spfile string +DATA/rac/spfilerac.ora

SQL>

SQL> show parameter control_files

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

control_files string +DATA/rac/controlfile/current.

256.876411235 SQL>

SQL> select file_name from dba_data_files;

FILE_NAME

--------------------------------------------------

+DATA/rac/datafile/system.259.876411237

+DATA/rac/datafile/sysaux.260.876411239

+DATA/rac/datafile/undotbs1.261.876411241

+DATA/rac/datafile/undotbs2.263.876411247

+DATA/rac/datafile/users.264.876411247

SQL>

在升级前需要对数据文件、控制文件以及参数文件进行冷备

5.9 备份数据库

数据库全备份，如下：

connected to target database: RAC (DBID=2464393249, not open)

RMAN> backup database plus archivelog delete input format '/s01/soft/full_%s_%p_%t_%T.bak';

Starting backup at 07-APR-15

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=38 instance=rac1 device type=DISK

channel ORA_DISK_1: starting archived log backup set

channel ORA_DISK_1: specifying archived log(s) in backup set

input archived log thread=1 sequence=65 RECID=2 STAMP=876413675

input archived log thread=2 sequence=1 RECID=1 STAMP=876413524

input archived log thread=2 sequence=2 RECID=4 STAMP=876415614

input archived log thread=1 sequence=66 RECID=3 STAMP=876413675

input archived log thread=2 sequence=3 RECID=5 STAMP=876415614

input archived log thread=1 sequence=67 RECID=8 STAMP=876415617

input archived log thread=2 sequence=4 RECID=6 STAMP=876415614

input archived log thread=2 sequence=5 RECID=7 STAMP=876415614

input archived log thread=1 sequence=68 RECID=9 STAMP=876416049

channel ORA_DISK_1: starting piece 1 at 07-APR-15

channel ORA_DISK_1: finished piece 1 at 07-APR-15

piece handle=/s01/soft/full_1_1_876417362_20150407.bak tag=TAG20150407T171602 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

channel ORA_DISK_1: deleting archived log(s)

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_1_seq_65.269.876413675 RECID=2 STAMP=876413675

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_2_seq_1.268.876413525 RECID=1 STAMP=876413524

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_2_seq_2.271.876415613 RECID=4 STAMP=876415614

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_1_seq_66.270.876413675 RECID=3 STAMP=876413675

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_2_seq_3.272.876415615 RECID=5 STAMP=876415614

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_1_seq_67.275.876415617 RECID=8 STAMP=876415617

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_2_seq_4.273.876415615 RECID=6 STAMP=876415614

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_2_seq_5.274.876415615 RECID=7 STAMP=876415614

archived log file name=+DATA/rac/archivelog/2015_04_07/thread_1_seq_68.276.876416049 RECID=9 STAMP=876416049

Finished backup at 07-APR-15

Starting backup at 07-APR-15

using channel ORA_DISK_1

channel ORA_DISK_1: starting full datafile backup set

channel ORA_DISK_1: specifying datafile(s) in backup set

input datafile file number=00003 name=+DATA/rac/datafile/undotbs1.261.876411241

input datafile file number=00001 name=+DATA/rac/datafile/system.259.876411237

input datafile file number=00002 name=+DATA/rac/datafile/sysaux.260.876411239

input datafile file number=00004 name=+DATA/rac/datafile/undotbs2.263.876411247

input datafile file number=00005 name=+DATA/rac/datafile/users.264.876411247

channel ORA_DISK_1: starting piece 1 at 07-APR-15

channel ORA_DISK_1: finished piece 1 at 07-APR-15

piece handle=+DATA/rac/backupset/2015_04_07/nnndf0_tag20150407t171604_0.276.876417365 tag=TAG20150407T171604 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:15

channel ORA_DISK_1: starting full datafile backup set

channel ORA_DISK_1: specifying datafile(s) in backup set

including current control file in backup set

including current SPFILE in backup set

channel ORA_DISK_1: starting piece 1 at 07-APR-15

channel ORA_DISK_1: finished piece 1 at 07-APR-15

piece handle=+DATA/rac/backupset/2015_04_07/ncsnf0_tag20150407t171604_0.274.876417383 tag=TAG20150407T171604 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

Finished backup at 07-APR-15

Starting backup at 07-APR-15

using channel ORA_DISK_1

specification does not match any archived log in the repository

backup cancelled because there are no files to backup

Finished backup at 07-APR-15

RMAN> BACKUP FORMAT '/s01/soft/control_%s_%p_%t_%T' CURRENT CONTROLFILE;

Starting backup at 07-APR-15

using channel ORA_DISK_1

channel ORA_DISK_1: starting full datafile backup set

channel ORA_DISK_1: specifying datafile(s) in backup set

including current control file in backup set

channel ORA_DISK_1: starting piece 1 at 07-APR-15

channel ORA_DISK_1: finished piece 1 at 07-APR-15

piece handle=/s01/soft/control_4_1_876417385_20150407 tag=TAG20150407T171625 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

Finished backup at 07-APR-15

5.10 备份GRID和数据库软件

通过tar方式备份ORACLE软件目录

[root@ol112301 /]# pwd

[root@ol112301 /]# tar -cvf g101.tar g01

[root@ol112301 /]# tar -cvf s101.tar s01

[root@ol112302 /]# pwd

[root@ol112302 /]# tar -cvf g201.tar g01

[root@ol112302 /]# tar -cvf s201.tar s01

升级GRID软件

6.1关闭数据库

关闭数据库即可

[oracle@ol112301 ~]$ srvctl stop database -d rac

6.2 安装11.2.0.4.0版GI软件

在新GI目录里面安装11.2.0.4.0版GRID软件目录, 新GI目录准备命令如下， 2个节点均用root用户执行：

mkdir -p /ng01/11ggrid/app/11.2.0/grid

mkdir /ng01/orabase

chown -R grid:oinstall /ng01

chmod -R 775 /ng01

解压GI补丁包

[grid@ol112301 grid]$ unzip p19955028_112040_Linux-x86-64.zip

执行安装

[grid@ol112301 grid]$ export DISPLAY=192.168.1.103:0.0

[grid@ol112301 grid]$ ./runInstaller

注意如下的software location项，软件默认识别出来的变量可能并不正确，其应该是填写11.2.0.4.0的$GI_HOME,

节点1上用root用户执行/ng01/11ggrid/app/11.2.0/grid/rootupgrade.sh，日志如下：

[root@ol112301 ~]# /ng01/11ggrid/app/11.2.0/grid/rootupgrade.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /ng01/11ggrid/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /ng01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

Installing Trace File Analyzer

ASM upgrade has started on first node.

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol112301'

CRS-2673: Attempting to stop 'ora.crsd' on 'ol112301'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ol112301'

CRS-2673: Attempting to stop 'ora.DATA.dg' on 'ol112301'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'ol112301'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'ol112301'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.ol112301.vip' on 'ol112301'

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'ol112301'

CRS-2677: Stop of 'ora.ol112301.vip' on 'ol112301' succeeded

CRS-2672: Attempting to start 'ora.ol112301.vip' on 'ol112302'

CRS-2677: Stop of 'ora.scan1.vip' on 'ol112301' succeeded

CRS-2672: Attempting to start 'ora.scan1.vip' on 'ol112302'

CRS-2676: Start of 'ora.ol112301.vip' on 'ol112302' succeeded

CRS-2676: Start of 'ora.scan1.vip' on 'ol112302' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'ol112302'

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'ol112302' succeeded

CRS-2677: Stop of 'ora.DATA.dg' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'ol112301'

CRS-2677: Stop of 'ora.asm' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'ol112301'

CRS-2677: Stop of 'ora.ons' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'ol112301'

CRS-2677: Stop of 'ora.net1.network' on 'ol112301' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'ol112301' has completed

CRS-2677: Stop of 'ora.crsd' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'ol112301'

CRS-2673: Attempting to stop 'ora.evmd' on 'ol112301'

CRS-2673: Attempting to stop 'ora.asm' on 'ol112301'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'ol112301'

CRS-2677: Stop of 'ora.evmd' on 'ol112301' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'ol112301' succeeded

CRS-2677: Stop of 'ora.ctssd' on 'ol112301' succeeded

CRS-2677: Stop of 'ora.asm' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ol112301'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'ol112301'

CRS-2677: Stop of 'ora.cssd' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'ol112301'

CRS-2677: Stop of 'ora.crf' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'ol112301'

CRS-2677: Stop of 'ora.gipcd' on 'ol112301' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'ol112301'

CRS-2677: Stop of 'ora.gpnpd' on 'ol112301' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol112301' has completed

CRS-4133: Oracle High Availability Services has been stopped.

OLR initialization - successful

Replacing Clusterware entries in inittab

clscfg: EXISTING configuration version 5 detected.

clscfg: version 5 is 11g Release 2.

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

节点2上用root用户执行/ng01/11ggrid/app/11.2.0/grid/rootupgrade.sh，日志如下：

[root@ol112302 ~]# /ng01/11ggrid/app/11.2.0/grid/rootupgrade.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /ng01/11ggrid/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /ng01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

Installing Trace File Analyzer

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ol112302'

CRS-2673: Attempting to stop 'ora.crsd' on 'ol112302'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ol112302'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'ol112302'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'ol112302'

CRS-2673: Attempting to stop 'ora.DATA.dg' on 'ol112302'

CRS-2673: Attempting to stop 'ora.cvu' on 'ol112302'

CRS-2677: Stop of 'ora.cvu' on 'ol112302' succeeded

CRS-2672: Attempting to start 'ora.cvu' on 'ol112301'

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'ol112302'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.ol112302.vip' on 'ol112302'

CRS-2677: Stop of 'ora.scan1.vip' on 'ol112302' succeeded

CRS-2672: Attempting to start 'ora.scan1.vip' on 'ol112301'

CRS-2677: Stop of 'ora.ol112302.vip' on 'ol112302' succeeded

CRS-2672: Attempting to start 'ora.ol112302.vip' on 'ol112301'

CRS-2676: Start of 'ora.cvu' on 'ol112301' succeeded

CRS-2676: Start of 'ora.scan1.vip' on 'ol112301' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'ol112301'

CRS-2676: Start of 'ora.ol112302.vip' on 'ol112301' succeeded

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'ol112301' succeeded

CRS-2677: Stop of 'ora.DATA.dg' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'ol112302'

CRS-2677: Stop of 'ora.asm' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'ol112302'

CRS-2677: Stop of 'ora.ons' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'ol112302'

CRS-2677: Stop of 'ora.net1.network' on 'ol112302' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'ol112302' has completed

CRS-2677: Stop of 'ora.crsd' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'ol112302'

CRS-2673: Attempting to stop 'ora.ctssd' on 'ol112302'

CRS-2673: Attempting to stop 'ora.evmd' on 'ol112302'

CRS-2673: Attempting to stop 'ora.asm' on 'ol112302'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'ol112302'

CRS-2677: Stop of 'ora.crf' on 'ol112302' succeeded

CRS-2677: Stop of 'ora.evmd' on 'ol112302' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'ol112302' succeeded

CRS-2677: Stop of 'ora.ctssd' on 'ol112302' succeeded

CRS-2677: Stop of 'ora.asm' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ol112302'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'ol112302'

CRS-2677: Stop of 'ora.cssd' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'ol112302'

CRS-2677: Stop of 'ora.gipcd' on 'ol112302' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'ol112302'

CRS-2677: Stop of 'ora.gpnpd' on 'ol112302' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ol112302' has completed

CRS-4133: Oracle High Availability Services has been stopped.

OLR initialization - successful

Replacing Clusterware entries in inittab

clscfg: EXISTING configuration version 5 detected.

clscfg: version 5 is 11g Release 2.

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Started to upgrade the Oracle Clusterware. This operation may take a few minutes.

Started to upgrade the CSS.

Started to upgrade the CRS.

The CRS was successfully upgraded.

Successfully upgraded the Oracle Clusterware.

Oracle Clusterware operating version was successfully set to 11.2.0.4.0

ASM upgrade has finished on last node.

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

执行完脚本后，点击下图的ok继续，

如上2个错误不影响正常的升级。

GRID软件升级完成。

6.3 检查升级后的版本

[grid@ol112301 soft]$ /ng01/11ggrid/app/11.2.0/grid/bin/crsctl query crs releaseversion

Oracle High Availability Services release version on the local node is [11.2.0.4.0] [grid@ol112301 soft]$ /ng01/11ggrid/app/11.2.0/grid/bin/crsctl query crs softwareversion

Oracle Clusterware version on node [ol112301] is [11.2.0.4.0]

[grid@ol112301 soft]$ /ng01/11ggrid/app/11.2.0/grid/bin/crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [11.2.0.4.0]

[grid@ol112302 soft]$ /ng01/11ggrid/app/11.2.0/grid/bin/crsctl query crs releaseversion

Oracle High Availability Services release version on the local node is [11.2.0.4.0] [grid@ol112302 soft]$ /ng01/11ggrid/app/11.2.0/grid/bin/crsctl query crs softwareversion

Oracle Clusterware version on node [ol112302] is [11.2.0.4.0]

[grid@ol112302 soft]$ /ng01/11ggrid/app/11.2.0/grid/bin/crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [11.2.0.4.0]

6.4 修改GRID用户环境变量

2个节点grid用户的变量都需要修改为11.2.0.4.0版的目录

#export ORACLE_HOME=/g01/11ggrid/app/11.2.0/grid

export ORACLE_HOME=/ng01/11ggrid/app/11.2.0/grid

升级数据库软件

7.1 创建软件安装目录

在新DB目录里面安装11.2.0.4.0版数据库软件, 分别在2个节点用root用户执行如下命令:

mkdir -p /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

chown -R oracle:oinstall /ns01

chmod -R 775 /ns01

屏蔽ORACLE环境变量

屏蔽2个节点中ORACLE用户下的.bash_profile中的$ORACLE_BASE, $ORACLE_HOME,

$ORACLE_SID

7.3 关闭数据库

关闭数据库进行数据库升级，各资源状态如下：

[oracle@ol112301 ~]$ srvctl stop database -d rac

[root@ol112301 ~]# crsctl stat res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.DATA.dg

ONLINE ONLINE ol112301

ONLINE ONLINE ol112302

ora.LISTENER.lsnr

ONLINE ONLINE ol112301

ONLINE ONLINE ol112302

ora.asm

ONLINE ONLINE ol112301 Started

ONLINE ONLINE ol112302 Started

ora.gsd

OFFLINE OFFLINE ol112301

OFFLINE OFFLINE ol112302

ora.net1.network

ONLINE ONLINE ol112301

ONLINE ONLINE ol112302

ora.ons

ONLINE ONLINE ol112301

ONLINE ONLINE ol112302

ora.registry.acfs

ONLINE ONLINE ol112301

ONLINE ONLINE ol112302

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE ol112301

ora.cvu

1 ONLINE ONLINE ol112301

ora.oc4j

1 ONLINE ONLINE ol112302

ora.ol112301.vip

1 ONLINE ONLINE ol112301

ora.ol112302.vip

1 ONLINE ONLINE ol112302

ora.rac.db

1 OFFLINE OFFLINE

2 OFFLINE OFFLINE Instance Shutdown

ora.scan1.vip

1 ONLINE ONLINE ol112301

7.4 解压数据库软件文件

unzip p13390677_112040_Linux-x86-64_1of7.zip

unzip p13390677_112040_Linux-x86-64_2of7.zip

7.5 开始升级数据库软件

执行安装

[oracle@ol112301 database]$ export DISPLAY=192.168.1.103:0.0

[oracle@ol112301 database]$ ./runInstaller

选择YES

注意将ORACLE_BASE, software location修改为11.2.0.4.0版的环境变量，如下：

选择YES.

节点1 root用户执行root.sh

[root@ol112301 ~]# /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

节点2 root用户执行root.sh

[root@ol112302 ~]# /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

You have new mail in /var/spool/mail/root

数据库软件升级完成。

8 升级数据库

8.1 启动11.2.0.3.0版数据库

[oracle@ol112301 ~]$ export ORACLE_HOME=/s01/oracle/app/oracle/product/11.2.0/dbhome_1

[oracle@ol112301 ~]$ export ORACLE_SID=rac1

[oracle@ol112301 ~]$ $ORACLE_HOME/bin/srvctl status database -d rac

Instance rac1 is not running on node ol112301

Instance rac2 is not running on node ol112302

[oracle@ol112301 ~]$ $ORACLE_HOME/bin/srvctl start database -d rac

[oracle@ol112301 ~]$ $ORACLE_HOME/bin/srvctl status database -d rac

Instance rac1 is running on node ol112301

Instance rac2 is running on node ol112302

8.2数据库升级前检查

使用11.2.0.4.0版的$ORACLE_HOME/ rdbms/admin/utlu112i.sql脚本进行检查，信息如下：

[oracle@ol112301 ~]$ $ORACLE_HOME/bin/sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue Apr 7 10:16:29 2015

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

SQL> @/ns01/oracle/app/oracle/product/11.2.0/dbhome_1/rdbms/admin/utlu112i.sql

Oracle Database 11.2 Pre-Upgrade Information Tool 04-07-2015 18:38:55

Script Version: 11.2.0.4.0 Build: 001

**********************************************************************

Database:

**********************************************************************

--> name: RAC

--> version: 11.2.0.3.0

--> compatible: 11.2.0.0.0

--> blocksize: 8192

--> platform: Linux x86 64-bit

--> timezone file: V14

**********************************************************************

Tablespaces: [make adjustments in the current environment]

**********************************************************************

--> SYSTEM tablespace is adequate for the upgrade.

.... minimum required size: 887 MB

--> SYSAUX tablespace is adequate for the upgrade.

.... minimum required size: 569 MB

--> UNDOTBS1 tablespace is adequate for the upgrade.

.... minimum required size: 400 MB

--> TEMP tablespace is adequate for the upgrade.

.... minimum required size: 60 MB

**********************************************************************

Flashback: ON

**********************************************************************

FlashbackInfo:

--> name: +DATA

--> limit: 6144 MB

--> used: 1868 MB

--> size: 6144 MB

--> reclaim: 0 MB

--> files: 7

WARNING: --> Flashback Recovery Area Set. Please ensure adequate disk space in recover

y areas before performing an upgrade.

**********************************************************************

Update Parameters: [Update Oracle Database 11.2 init.ora or spfile]

Note: Pre-upgrade tool was run on a lower version 64-bit database.

**********************************************************************

--> If Target Oracle is 32-Bit, refer here for Update Parameters:

-- No update parameter changes are required.

--> If Target Oracle is 64-Bit, refer here for Update Parameters:

-- No update parameter changes are required.

**********************************************************************

Renamed Parameters: [Update Oracle Database 11.2 init.ora or spfile]

**********************************************************************

-- No renamed parameters found. No changes are required.

**********************************************************************

Obsolete/Deprecated Parameters: [Update Oracle Database 11.2 init.ora or spfile]

**********************************************************************

-- No obsolete parameters found. No changes are required

**********************************************************************

Components: [The following database components will be upgraded or installed]

**********************************************************************

--> Oracle Catalog Views [upgrade] VALID

--> Oracle Packages and Types [upgrade] VALID

--> JServer JAVA Virtual Machine [upgrade] VALID

--> Oracle XDK for Java [upgrade] VALID

--> Real Application Clusters [upgrade] VALID

--> Oracle Workspace Manager [upgrade] VALID

--> EM Repository [upgrade] VALID

--> Oracle Text [upgrade] VALID

--> Oracle XML Database [upgrade] VALID

--> Oracle Java Packages [upgrade] VALID

--> Oracle interMedia [upgrade] VALID

--> Spatial [upgrade] VALID

--> Expression Filter [upgrade] VALID

--> Rule Manager [upgrade] VALID

--> Oracle Application Express [upgrade] VALID

... APEX will only be upgraded if the version of APEX in

... the target Oracle home is higher than the current one.

**********************************************************************

Miscellaneous Warnings

**********************************************************************

WARNING: --> The "cluster_database" parameter is currently "TRUE"

.... and must be set to "FALSE" prior to running a manual upgrade.

WARNING: --> Your recycle bin contains 2 object(s).

.... It is REQUIRED that the recycle bin is empty prior to upgrading

.... your database. The command:

PURGE DBA_RECYCLEBIN

.... must be executed immediately prior to executing your upgrade.

WARNING: --> Database contains schemas with objects dependent on DBMS_LDAP package.

.... Refer to the 11g Upgrade Guide for instructions to configure Network ACLs.

.... USER APEX_030200 has dependent objects.

**********************************************************************

Recommendations

**********************************************************************

Oracle recommends gathering dictionary statistics prior to

upgrading the database.

To gather dictionary statistics execute the following command

while connected as SYSDBA:

EXECUTE dbms_stats.gather_dictionary_stats;

**********************************************************************

Oracle recommends reviewing any defined events prior to upgrading.

To view existing non-default events execute the following commands

while connected AS SYSDBA:

Events:

SELECT (translate(value,chr(13)||chr(10),' ')) FROM sys.v$parameter2

WHERE UPPER(name) ='EVENT' AND isdefault='FALSE'

Trace Events:

SELECT (translate(value,chr(13)||chr(10),' ')) from sys.v$parameter2

WHERE UPPER(name) = '_TRACE_EVENTS' AND isdefault='FALSE'

Changes will need to be made in the init.ora or spfile.

**********************************************************************

解决上面提到的问题, 命令如下：

SQL> PURGE DBA_RECYCLEBIN;

SQL> EXECUTE dbms_stats.gather_dictionary_stats;

8.3 DBUA升级数据库

运行11.2.0.4的DBUA

[oracle@ol112301 ~]$ export DISPLAY=192.168.1.103:0.0

[oracle@ol112301 ~]$ /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/bin/dbua

选择YES

上图为升级数据库项的信息。

8.4 修改oracle环境变量

修改oracle用户的环境变量到新的ORACLE_HOME

export ORACLE_BASE=/ns01/oracle/app

export ORACLE_HOME=/ns01/oracle/app/oracle/product/11.2.0/dbhome_1

8.5 升级后的数据库版本

SQL> select * from v$version;

BANNER

--------------------------------------------------------------------------------

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

PL/SQL Release 11.2.0.4.0 - Production

CORE 11.2.0.4.0 Production

TNS for Linux: Version 11.2.0.4.0 - Production

NLSRTL Version 11.2.0.4.0 - Production

8.6 升级后的数据库注册组件

SQL> set lines 200 pages 9999

SQL> col COMP_NAME for a50

SQL> select comp_name,version,status from dba_registry;

COMP_NAME VERSION STATUS

-------------------------------------------------- ------------------------------ ----------------------

Oracle Application Express 3.2.1.00.12 VALID

Oracle Enterprise Manager 11.2.0.4.0 VALID

Spatial 11.2.0.4.0 VALID

Oracle Multimedia 11.2.0.4.0 VALID

Oracle XML Database 11.2.0.4.0 VALID

Oracle Text 11.2.0.4.0 VALID

Oracle Expression Filter 11.2.0.4.0 VALID

Oracle Rules Manager 11.2.0.4.0 VALID

Oracle Workspace Manager 11.2.0.4.0 VALID

Oracle Database Catalog Views 11.2.0.4.0 VALID

Oracle Database Packages and Types 11.2.0.4.0 VALID

JServer JAVA Virtual Machine 11.2.0.4.0 VALID

Oracle XDK 11.2.0.4.0 VALID

Oracle Database Java Packages 11.2.0.4.0 VALID

Oracle Real Application Clusters 11.2.0.4.0 VALID

15 rows selected.

8.7 升级后的软件版本和兼容性

SQL> col INSTANCE_NAME for a15

SQL> col SOFTWARE_VERSION for a15

SQL> col COMPATIBLE_VERSION for a15

SQL> set lines 200 pages 999

SQL> select inst_id,instance_name,software_version,compatible_version from gv$asm_client where status = 'CONNECTED';

INST_ID INSTANCE_NAME SOFTWARE_VERSION COMPATIBLE_VERSION

---------- ---------------------------------------------------------------- ------------------------------------------------------------ ------------------------------------------------------------

1 +ASM1 11.2.0.4.0 11.2.0.0.0

2 +ASM2 11.2.0.4.0 11.2.0.0.0

8.8 升级后的无效对象

SQL> set lines 200 pages 999

SQL> Col owner for a30

SQL> Col object_name for a40

SQL> Col object_type for a30

SQL>SELECT owner, object_name, object_type, status

2 FROM dba_objects

3 WHERE status <> 'VALID'

4 ORDER BY owner, object_name;

no rows selected

升级后，数据库未产生失效的对象，情况和升级前一致。

9 升级打补丁工具

9.1 升级opatch

11.2.0.4.0版本的GI和DB软件的opatch的版本是11.2.0.3.4，但是升级到11.2.0.4.5要求的opatch版本为11.2.0.3.6及以上，未升级前的opatch版本如下：

[oracle@ol112301 ~]$ opatch version

OPatch Version: 11.2.0.3.4

备份任意1个节点的opatch即可，删除所有节点中旧的opatch，然后升级opatch，2个节点的oracle和grid用户都需要执行删除、升级操作

[root@ol112301 grid]# tar -cvf OPatch.old.tar OPatch

[root@ol112301 grid]# rm -rf OPatch

解压新版的OPATCH，2个节点的oracle和grid用户都需要执行

[grid@ol112301 grid]$ unzip p6880880_112000_Linux-x86-64.zip

验证opatch版本

[grid@ol112301 ~]$ which opatch

/ng01/11ggrid/app/11.2.0/grid/OPatch/opatch

[grid@ol112301 OPatch]$ opatch version

OPatch Version: 11.2.0.3.6

OPatch succeeded.

10 升级PSU

升级PSU到11.2.0.4.5，补丁号为19955028

10.1 解压GI PSU补丁

2个节点都需要上传，解压。

[grid@ol112301 soft]$ unzip p19955028_112040_Linux-x86-64.zip

10.2 创建OCM文件

1号节点

[grid@ol112301 bin]$ ./emocmrsp -no_banner -output /ng01/soft/ocm.rsp

Provide your email address to be informed of security issues, install and

initiate Oracle Configuration Manager. Easier for you if you use your My

Oracle Support Email address/User Name.

Visit http://www.oracle.com/support/policies.html for details.

Email address/User Name:

You have not provided an email address for notification of security issues.

Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y

The OCM configuration response file (/ng01/soft/ocm.rsp) was successfully created.

[root@ol112302 ng01]# chmod -R 777 soft

---2号节点

[grid@ol112302 bin]$ ./emocmrsp -no_banner -output /ng01/soft/ocm.rsp

Provide your email address to be informed of security issues, install and

initiate Oracle Configuration Manager. Easier for you if you use your My

Oracle Support Email address/User Name.

Visit http://www.oracle.com/support/policies.html for details.

Email address/User Name:

You have not provided an email address for notification of security issues.

Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y

The OCM configuration response file (/ng01/soft/ocm.rsp) was successfully created.

[root@ol112302 ng01]# chmod -R 777 soft

10.3升级PSU（11.2.0.4.5）

查看EM是否启动，若启动则需要停止:

[oracle@ol112301 ~]$ emctl status dbconsole

Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0

https://ol112301:1158/em/console/aboutApplication

Oracle Enterprise Manager 11g is running.

------------------------------------------------------------------

Logs are generated in directory /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/ol112301_rac/sysman/log

[oracle@ol112301 ~]$ emctl stop dbconsole

Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0

https://ol112301:1158/em/console/aboutApplication

Stopping Oracle Enterprise Manager 11g Database Control ...

... Stopped.

[oracle@ol112302 ~]$ emctl status dbconsole

Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0

https://ol112301:1158/em/console/aboutApplication

EM Daemon is running.

------------------------------------------------------------------

Logs are generated in directory /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/ol112302_rac/sysman/log

[oracle@ol112302 ~]$ emctl stop dbconsole

Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0

https://ol112301:1158/em/console/aboutApplication

Stopping Oracle Enterprise Manager 11g Database Control ...

... Stopped.

为1号数据库实例打PSU补丁，如下：

[root@ol112301 ~]# which opatch

/ng01/11ggrid/app/11.2.0/grid/OPatch/opatch

[root@ol112301 ~]# opatch auto /ng01/soft/19955028 -ocmrf /ng01/soft/ocm.rsp

Executing /ng01/11ggrid/app/11.2.0/grid/perl/bin/perl /ng01/11ggrid/app/11.2.0/grid/OPatch/crs/patch11203.pl -patchdir /ng01/soft -patchn 19955028 -ocmrf /ng01/soft/ocm.rsp -paramfile /ng01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params

This is the main log file: /ng01/11ggrid/app/11.2.0/grid/cfgtoollogs/opatchauto2015-04-07_20-35-20.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:

/ng01/11ggrid/app/11.2.0/grid/cfgtoollogs/opatchauto2015-04-07_20-35-20.report.log

2015-04-07 20:35:20: Starting Clusterware Patch Setup

Using configuration parameter file: /ng01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params

Stopping RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 ...

Stopped RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 successfully

patch /ng01/soft/19955028/19769489 apply successful for home /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

patch /ng01/soft/19955028/19769476/custom/server/19769476 apply successful for home /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

Stopping CRS...

Stopped CRS successfully

patch /ng01/soft/19955028/19769489 apply successful for home /ng01/11ggrid/app/11.2.0/grid

patch /ng01/soft/19955028/19769476 apply successful for home /ng01/11ggrid/app/11.2.0/grid

patch /ng01/soft/19955028/19769469 apply successful for home /ng01/11ggrid/app/11.2.0/grid

Starting CRS...

Installing Trace File Analyzer

CRS-4123: Oracle High Availability Services has been started.

Starting RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 ...

Started RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 successfully

opatch auto succeeded.

为2号数据库实例打PSU补丁，如下：

[root@ol112302 ~]# which opatch

/ng01/11ggrid/app/11.2.0/grid/OPatch/opatch

[root@ol112302 ~]# opatch auto /ng01/soft/19955028 -ocmrf /ng01/soft/ocm.rsp

This is the main log file: /ng01/11ggrid/app/11.2.0/grid/cfgtoollogs/opatchauto2015-04-08_11-21-02.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:

/ng01/11ggrid/app/11.2.0/grid/cfgtoollogs/opatchauto2015-04-08_11-21-02.report.log

2015-04-08 11:21:02: Starting Clusterware Patch Setup

Using configuration parameter file: /ng01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params

Stopping RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 ...

Stopped RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 successfully

patch /ng01/soft/19955028/19769489 apply successful for home /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

patch /ng01/soft/19955028/19769476/custom/server/19769476 apply successful for home /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

Stopping CRS...

Stopped CRS successfully

patch /ng01/soft/19955028/19769489 apply successful for home /ng01/11ggrid/app/11.2.0/grid

patch /ng01/soft/19955028/19769476 apply successful for home /ng01/11ggrid/app/11.2.0/grid

patch /ng01/soft/19955028/19769469 apply successful for home /ng01/11ggrid/app/11.2.0/grid

Starting CRS...

Installing Trace File Analyzer

CRS-4123: Oracle High Availability Services has been started.

Starting RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 ...

Started RAC /ns01/oracle/app/oracle/product/11.2.0/dbhome_1 successfully

opatch auto succeeded.

验证安装好的PSU情况:

1号数据库节点GRID用户下的补丁情况:

[grid@ol112301 ~]$ opatch lsinventory

Oracle Interim Patch Installer version 11.2.0.3.6

Oracle Home : /ng01/11ggrid/app/11.2.0/grid

Central Inventory : /g01/oraInventory

from : /ng01/11ggrid/app/11.2.0/grid/oraInst.loc

OPatch version : 11.2.0.3.6

OUI version : 11.2.0.4.0

Log file location : /ng01/11ggrid/app/11.2.0/grid/cfgtoollogs/opatch/opatch2015-04-08_11-36-03AM_1.log

Lsinventory Output file location : /ng01/11ggrid/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2015-04-08_11-36-03AM.txt

--------------------------------------------------------------------------------

Installed Top-level Products (1):

Oracle Grid Infrastructure 11g 11.2.0.4.0

There are 1 product(s) installed in this Oracle Home.

Interim patches (3) :

Patch 19769469 : applied on Tue Apr 07 20:44:00 CST 2015

Unique Patch ID: 18264903

Patch description: "ACFS Patch Set Update : 11.2.0.4.5 (19769469)"

Created on 13 Nov 2014, 04:37:23 hrs PST8PDT

Bugs fixed:

17510275, 16318126, 17172303, 19690653, 17203009, 17376318, 17503605

17611362, 17721778, 17164243, 17696547, 19053182, 17699423, 17488768

18915417, 18155334, 19919907, 18168684, 17363999, 18143006, 17636008

17428148, 17070158, 17475946

Patch 19769476 : applied on Tue Apr 07 20:43:07 CST 2015

Unique Patch ID: 18264903

Patch description: "OCW Patch Set Update : 11.2.0.4.5 (19769476)"

Created on 3 Dec 2014, 02:08:05 hrs PST8PDT

Bugs fixed:

19270660, 18328800, 18691572, 14525998, 18187697, 14385860, 19479503

18348155, 18370031, 17516024, 17387214, 17750548, 17617807, 14497275

17764053, 17551223, 14671408, 17733927, 14207615, 18272135, 18180541

18962892, 17292250, 17378618, 17500165, 16759171, 18875012, 19558324

18464784, 17065496, 18848125, 13991403, 19241857, 17955615, 14693336

17273020, 17238586, 17089344, 17405605, 17531342, 17155238, 17640316

17159489, 18261183, 18053580, 16543190, 17039197, 16317771, 17947785

10052729, 16281493, 18346135, 17481314, 18199185, 18700935, 18399991

14076173, 18024089, 18428146, 17435488, 18352845, 18352846, 17387779

17391726, 18414137, 17001914, 17927970, 14378120, 16346413, 17305100

15832129, 15986647, 16901346, 18068871, 17985714, 18536826, 17780903

16206997, 18752378, 18946768, 16875342, 16876500, 19168690, 16429265

18343490, 18336452, 16613232, 17273003, 19276791, 17722664, 12928658

18226143, 17059927, 17172091, 18229842, 18053631, 16867761, 18231837

15869775, 14373486, 19642566, 17483479, 18729166, 17405302, 15920201, 18709496

Patch 19769489 : applied on Tue Apr 07 20:42:10 CST 2015

Unique Patch ID: 18236413

Patch description: "Database Patch Set Update : 11.2.0.4.5 (19769489)"

Created on 28 Dec 2014, 21:22:44 hrs PST8PDT

Sub-patch 19121551; "Database Patch Set Update : 11.2.0.4.4 (19121551)"

Sub-patch 18522509; "Database Patch Set Update : 11.2.0.4.3 (18522509)"

Sub-patch 18031668; "Database Patch Set Update : 11.2.0.4.2 (18031668)"

Sub-patch 17478514; "Database Patch Set Update : 11.2.0.4.1 (17478514)"

Bugs fixed:

17288409, 18273830, 17811429, 17205719, 19972566, 17922254, 17754782

17726838, 16934803, 13364795, 17311728, 17441661, 17284817, 13645875

16992075, 18199537, 16542886, 17446237, 14565184, 19972569, 17071721

17610798, 17449815, 17375354, 17397545, 19463897, 18230522, 13866822

16360112, 17982555, 17235750, 19769489, 17478514, 12905058, 18235390

14338435, 13944971, 18641451, 20142975, 16929165, 12747740, 17546973

14054676, 17088068, 18264060, 17042658, 17343514, 17016369, 14602788

19972568, 19680952, 18471685, 18508861, 14657740, 17332800, 19211724

13951456, 16315398, 19972564, 18744139, 17186905, 16850630, 18315328

17437634, 19049453, 18673304, 17883081, 19006849, 18641419, 17296856

18674024, 18262334, 17006183, 18277454, 16833527, 17232014, 16855292

10136473, 17705023, 17865671, 18554871, 19121551, 18334586, 19854503

19309466, 17551709, 17588480, 19827973, 17842825, 17344412, 18828868

18681862, 18554763, 17390160, 17025461, 13955826, 11883252, 13609098

18139690, 17501491, 17239687, 17299889, 17752121, 17602269, 19197175

18673325, 17313525, 18293054, 17242746, 19544839, 18964939, 17600719

18191164, 17571306, 18482502, 19466309, 17951233, 18094246, 19615136

17165204, 17011832, 17040527, 18098207, 16785708, 17477958, 17174582

16180763, 17465741, 16777840, 18522509, 17323222, 19463893, 16595641

12816846, 16875449, 16524926, 17237521, 18228645, 18282562, 17596908

17811438, 17811447, 18031668, 17184721, 16494615, 16912439, 18061914

17282229, 17545847, 18202441, 17082359, 19554106, 17614134, 13558557

17341326, 14034426, 17891946, 18339044, 19458377, 17716305, 17752995

16392068, 19271443, 17891943, 18092127, 17258090, 17767676, 17614227

17040764, 14106803, 17381384, 15913355, 18973907, 18356166, 18673342

14084247, 17389192, 17612828, 17006570, 13853126, 17721717, 17570240

18203837, 17390431, 14245531, 16043574, 16863422, 19727057, 17468141

17786518, 17912217, 16422541, 19972570, 17037130, 17267114, 18244962

18203838, 16198143, 16956380, 17478145, 14829250, 17394950, 18189036

18641461, 17835627, 17027426, 16268425, 18247991, 14458214, 19584068

18436307, 17265217, 13498382, 16692232, 17786278, 17227277, 16042673

16314254, 17443671, 16228604, 16837842, 17571039, 17393683, 16344544

17787259, 18009564, 20074391, 14354737, 15861775, 18135678, 18614015

16399083, 18018515, 17835048, 16472716, 17936109, 17050888, 17325413

14010183, 18747196, 17036973, 16613964, 17080436, 17761775, 16721594

18280813, 17302277, 16901385, 15979965, 15990359, 18203835, 17297939

17811456, 16731148, 17215560, 14133975, 17385178, 17586955, 18441944

16450169, 17655634, 9756271, 19730508, 17892268, 17648596, 16220077

16069901, 11733603, 16285691, 17587063, 16538760, 18180390, 17393915

17274537, 18096714, 17308789, 17238511, 18436647, 17824637, 14285317

19289642, 14764829, 18328509, 17622427, 16943711, 18306996, 17346671

18996843, 14852021, 17783588, 16618694, 18674047, 17672719, 12364061

17851160, 17546761

Rac system comprising of multiple nodes

Local node = ol112301

Remote node = ol112302

--------------------------------------------------------------------------------

OPatch succeeded.

1号数据库节点ORACLE用户下的补丁情况:

[oracle@ol112301 ~]$ opatch lsinventory

Oracle Interim Patch Installer version 11.2.0.3.6

Oracle Home : /ns01/oracle/app/oracle/product/11.2.0/dbhome_1

Central Inventory : /g01/oraInventory

from : /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/oraInst.loc

OPatch version : 11.2.0.3.6

OUI version : 11.2.0.4.0

Log file location : /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2015-04-08_11-37-16AM_1.log

Lsinventory Output file location : /ns01/oracle/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2015-04-08_11-37-16AM.txt

--------------------------------------------------------------------------------

Installed Top-level Products (1):

Oracle Database 11g 11.2.0.4.0

There are 1 product(s) installed in this Oracle Home.

Interim patches (2) :

Patch 19769476 : applied on Tue Apr 07 20:38:36 CST 2015

Unique Patch ID: 18264903

Patch description: "OCW Patch Set Update : 11.2.0.4.5 (19769476)"

Created on 3 Dec 2014, 02:07:57 hrs PST8PDT