It’s time to upgrade Oracle Database 11g (11.2.0.4 in this article) to Oracle Database 12.1.0.2. In this article I will demostrate the Step by Step how to upgrade Oracle Grid/Database 11.2.0.4 to 12.1.0.2 on Exadata Database Machine X5-2 Full Rack.
Since the upgrade task is very lengthy, I will split the article in 2 different parts:
- Part I:
- Prepare the Current Exadata Environment &
- Install and Upgrade Grid Infrastructure to 12.1.0.2
- Part II:
- Install Database 12.1.0.2 Software &
- Upgrade Database to 12.1.0.2 using DBCA and Post Upgrade Steps
- Exadata Database Machine X5-2 Full Rack
- Exadata Storage Software version 12.1.2.1.1
- Oracle Grid/Database Version 11.2.0.4
Download Grid/Database Software and Copy it Exadata Compute node 1:
Follow the steps by step instruction below to download the Grid/Database software for upgrade
dm01db01-orcldb1 {/u01/app/oracle/software}:ls -l
-rw-r–r– 1 oracle oinstall 1673519571 Jan 9 00:30 p21419221_121020_Linux-x86-64_1of10.zip
-rw-r–r– 1 oracle oinstall 1014527110 Jan 9 00:30 p21419221_121020_Linux-x86-64_2of10.zip
-rw-r–r– 1 oracle oinstall 646969279 Jan 9 01:25 p21419221_121020_Linux-x86-64_6of10.zip
-rw-r–r– 1 oracle oinstall 1747021273 Jan 9 01:38 p21419221_121020_Linux-x86-64_5of10.zip
Note: There are 2 zip files each for Database and Grid Infrastructure.
- zip files 1of10 and 2of10 are Database specific files
- zip files 5of10 and 6of10 are Grid specific files
Unzip of both zip files in same directory as follows:
- Unzip database zip files
dm01db01-orcldb1 {/u01/app/oracle/software}:unzip p21419221_121020_Linux-x86-64_1of10.zip
Archive: p21419221_121020_Linux-x86-64_1of10.zip
creating: database/
inflating: database/runInstaller
creating: database/rpm/
inflating: database/rpm/cvuqdisk-1.0.9-1.rpm
creating: database/install/
inflating: database/install/attachHome.sh
inflating: database/install/oraparam.ini.deinstall
creating: database/install/images/
inflating: database/install/images/billboards.gif
creating: database/install/resource/
inflating: database/install/resource/cons.nls
…
inflating: database/stage/Actions/launchPadActions/10.1.0.2.0/1/launchpadaction.jar
creating: database/response/
inflating: database/response/netca.rsp
inflating: database/response/dbca.rsp
inflating: database/response/db_install.rsp
inflating: PatchSearch.xml
dm01db01-orcldb1 {/u01/app/oracle/software}:unzip p21419221_121020_Linux-x86-64_2of10.zip
Archive: p21419221_121020_Linux-x86-64_2of10.zip
creating: database/stage/Components/oracle.ctx/
creating: database/stage/Components/oracle.ctx/12.1.0.2.0/
creating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/
creating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/DataFiles/
inflating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/DataFiles/filegroup15.21.1.jar
inflating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/DataFiles/filegroup15.9.1.jar
inflating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/DataFiles/filegroup3.jar
inflating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/DataFiles/filegroup15.16.1.jar
inflating: database/stage/Components/oracle.ctx/12.1.0.2.0/1/DataFiles/filegroup4.jar
…
inflating: database/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup40.jar
creating: database/stage/Components/oracle.javavm.containers/
creating: database/stage/Components/oracle.javavm.containers/12.1.0.2.0/
creating: database/stage/Components/oracle.javavm.containers/12.1.0.2.0/1/
creating: database/stage/Components/oracle.javavm.containers/12.1.0.2.0/1/DataFiles/
inflating: database/stage/Components/oracle.javavm.containers/12.1.0.2.0/1/DataFiles/filegroup1.jar
inflating: database/stage/Components/oracle.javavm.containers/12.1.0.2.0/1/DataFiles/filegroup2.jar
inflating: database/install/.oui
- Unzip Grid Infrastructure zip files
dm01db01-orcldb1 {/u01/app/oracle/software}:unzip p21419221_121020_Linux-x86-64_5of10.zip
Archive: p21419221_121020_Linux-x86-64_5of10.zip
creating: grid/
inflating: grid/runcluvfy.sh
inflating: grid/welcome.html
creating: grid/install/
inflating: grid/install/oraparam.ini
inflating: grid/install/clusterparam.ini
inflating: grid/install/detachHome.sh
inflating: grid/install/cvu.ini
inflating: grid/install/lsnodes
creating: grid/install/resource/
inflating: grid/install/resource/cons_pt_BR.nls
inflating: grid/install/resource/cons_es.nls
inflating: grid/install/resource/cons_it.nls
inflating: grid/install/resource/cons.nls
inflating: grid/install/resource/cons_ko.nls
inflating: grid/install/resource/cons_zh_TW.nls
…
extracting: grid/stage/sizes/oracle.crs.Complete.sizes.properties
inflating: grid/stage/sizes/oracle.crs12.1.0.2.0Complete.sizes.properties
creating: grid/response/
inflating: grid/response/grid_install.rsp
creating: grid/sshsetup/
inflating: grid/sshsetup/sshUserSetup.sh
inflating: grid/runInstaller
inflating: grid/readme.html
creating: grid/rpm/
inflating: grid/rpm/cvuqdisk-1.0.9-1.rpm
dm01db01-orcldb1 {/u01/app/oracle/software}:unzip p21419221_121020_Linux-x86-64_6of10.zip
Archive: p21419221_121020_Linux-x86-64_6of10.zip
creating: grid/stage/Components/oracle.has.crs/
creating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/
creating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/
creating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/DataFiles/
inflating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/DataFiles/filegroup44.jar
inflating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/DataFiles/filegroup8.jar
inflating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/DataFiles/filegroup38.jar
inflating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/DataFiles/filegroup43.jar
inflating: grid/stage/Components/oracle.has.crs/12.1.0.2.0/1/DataFiles/filegroup31.jar
…
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup19.4.1.jar
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup73.jar
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup19.8.1.jar
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup4.jar
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup3.jar
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup19.22.1.jar
inflating: grid/stage/Components/oracle.rdbms/12.1.0.2.0/1/DataFiles/filegroup74.jar
inflating: grid/install/.oui
dm01db01-orcldb1 {/u01/app/oracle/software}:ls -ltr
total 4967820
drwxr-xr-x 7 oracle oinstall 4096 Jul 11 2014 database
drwxr-xr-x 7 oracle oinstall 4096 Jul 11 2014 grid
-rw-rw-r– 1 oracle oinstall 6584 Aug 13 2015 PatchSearch.xml
-rw-r–r– 1 oracle oinstall 1673519571 Jan 9 00:30 p21419221_121020_Linux-x86-64_1of10.zip
-rw-r–r– 1 oracle oinstall 1014527110 Jan 9 00:30 p21419221_121020_Linux-x86-64_2of10.zip
-rw-r–r– 1 oracle oinstall 646969279 Jan 9 01:25 p21419221_121020_Linux-x86-64_6of10.zip
-rw-r–r– 1 oracle oinstall 1747021273 Jan 9 01:38 p21419221_121020_Linux-x86-64_5of10.zip
Things to consider before upgrade to Oracle Grid Infrastructure 12c and Oracle Database 12c:
- Current Oracle Database and Grid Infrastructure version must be 11.2.0.2, 11.2.0.3, 11.2.0.4 or 12.1.0.1.
dm01db01-+ASM1 {/home/oracle}:sqlplus / as sysasm
SQL*Plus: Release 11.2.0.4.0 Production on Mon Jan 9 00:33:25 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> select * from v$version;
BANNER
——————————————————————————–
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
PL/SQL Release 11.2.0.4.0 – Production
CORE 11.2.0.4.0 Production
TNS for Linux: Version 11.2.0.4.0 – Production
NLSRTL Version 11.2.0.4.0 – Production
dm01db01-orcldb1 {/home/oracle}:sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Mon Jan 9 00:32:24 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> select * from v$version;
BANNER
——————————————————————————–
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
PL/SQL Release 11.2.0.4.0 – Production
CORE 11.2.0.4.0 Production
TNS for Linux: Version 11.2.0.4.0 – Production
NLSRTL Version 11.2.0.4.0 – Production
- Oracle Database 12c and Exadata
For full Exadata functionality including:
- Smart Scan offloaded filtering
- storage indexes
- I/O Resource Management (IORM),
Exadata Storage Server version 12.1.1.1.0 is required and 12.1.1.1.1 is recommended for Oracle Databases Release 12.1.0.2.
[root@dm01db01 ~]# imageinfo
Kernel version: 2.6.39-400.248.3.el6uek.x86_64 #1 SMP Wed Mar 11 18:04:34 PDT 2015 x86_64
Image version: 12.1.2.1.1.150316.2
Image activated: 2015-04-11 15:17:40 -0500
Image status: success
System partition on device: /dev/mapper/VGExaDb-LVDbSys1
Note: Customers unable to upgrade to Exadata 12.1.1.1.1 require a minimum version of 11.2.3.3.1 or later on Exadata Storage Servers and Database Servers. See document 1537407.1 for restrictions when running Oracle Database 12c in combination with Exadata releases earlier than 12.1.1.1.0.
- Sun Datacenter InfiniBand Switch 36 is running software release 1.3.3-2 or later, 2.1.3-4 recommended
[root@dm01db01 ~]# ssh dm01sw-ib1 version
root@dm01sw-ib1’s password:
SUN DCS 36p version: 2.1.5-1
Build time: Oct 6 2014 10:35:15
SP board info:
Manufacturing Date: 2012.09.12
Serial Number: “NCDA70501”
Hardware Revision: 0x0007
Firmware Revision: 0x0000
BIOS version: SUN0R100
BIOS date: 06/22/2010
In my case I am running ESS version 12.1.2.1.1 and IB Switch software version 2.1.5-1.
- Fix for bug 12539000 is required to successfully upgrade.
11.2.0.2 BP12 and later, 11.2.0.3, 11.2.0.4 and 12.1.0.1 already contain this fix. An interim patch must be installed for 11.2.0.2 Grid Infrastructure and Database installations running BP11 or earlier.
- Fix for bug 14639430 is required to properly rollback Grid Infrastructure upgrade, if necessary (not required when Grid Infrastructure already is on 11.2.0.4 or 12.1.0.1).
- Fix for bug 13460353 is required to create a new 11g database after Grid Infrastructure is upgraded to 12c.(not required when database already is on 11.2.0.4 or 12.1.0.1).
- When available: GI PSU 12.1.0.2.1 or later (which includes DB PSU 12.1.0.2.1). To be applied:
- during the upgrade process, before running rootupgrade.sh on the Grid Infrastructure home, or
- after installing the new Database home, before upgrading the database.
- Grid Infrastructure upgrades on nodes with different length hostnames in the same cluster require fix for bug 19453778 – CTSSD FAILED TO START WHILE RUNNING ROOTUPGRADE.SH. Contact Oracle Support to obtain the patch.
In my case I am upgrading from 11.2.0.4 to 12.1.0.2, these bugs are already fixed. So I don’t need to apply any of these Bug fixes.
High level steps involved in Oracle Grid/Database Upgrade process
- Pre-Upgrade steps: Preparing the current environment
- Install and Upgrade Oracle Grid Infrastructure to 12.1.0.2
- Install Oracle Database 12.1.0.2 Software
- Upgrade Oracle Database(s) to 12.1.0.2
- Post-upgrade steps
- Troubleshooting steps
Assumptions
- The Oracle Grid/Databasse software owner is oracle.
dm01db01-orcldb1 {/u01/app/oracle/software}:id oracle
uid=1000(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1003(oper),1004(asmdba)
- The Oracle inventory group is oinstall.
dm01db01-orcldb1 {/u01/app/oracle/software}:ls -ld /u01/app/oraInventory/
drwxrwx— 8 oracle oinstall 4096 Jan 8 04:44 /u01/app/oraInventory/
- Create dbs_group file in Oracle and Root users home directory and specify the all compute nodes one per line.
dm01db01-orcldb1 {/u01/app/oracle/software}:cat ~/dbs_group
dm01db01
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08
- Current database home. In my case database version is 11.2.0.4
dm01db01-orcldb1 {/u01/app/oracle/software}:echo $ORACLE_HOME
/u01/app/oracle/product/11.2.0.4/dbhome
- Current Grid Infrastructure home. In my case grid version is 11.2.0.4
dm01db01-+ASM1 {/u01/app/oracle/software}:echo $ORACLE_HOME
/u01/app/11.2.0.4/grid
- The primary database to be upgraded is named “orcldb”.
- Run Exadata Health Check (Exachk) and make sure no major hardware/software issues are reported.
Important MOS note:
Read them carefully before starting upgrade process:
- Document 888828.1 – Database Machine and Exadata Storage Server Supported Releases
- Document 1537407.1 – Requirements and restrictions when using Oracle Database 12c on Exadata Database Machine
- Document 1270094.1 – Exadata Critical Issues
- Document 1070954.1 – Oracle Exadata Database Machine exachk or HealthCheck
- Test the Upgrade on non-production environment first
- Ensure Oracle Grid/Database Homes are backed up before upgrade
[oracle@pr04db01 ~]# cd /u01/app/oracle/product/11.2.0.4/dbhome_1
[oracle@pr04db01 ~]# tar –zcvf /u01/app/oracle/product/11.2.0.4/db11204.tgz .
Grid Home backup:
[oracle@pr04db01 ~]# cd /u01/app/11.2.0.4/grid
[oracle@pr04db01 ~]# tar –zcvf /u01/app/11.2.0.4/gi11204.tgz .
- Optionally take a snapshot backup of /u01 file system
- Ensure the databases are backup before upgrade. Using RMAN backup to FRA or ZFS is recommended.
- It is recommended to create Guaranteed Restore Point in addition to Database backup
- Perform one-off patch assesment
- Do not place the new ORACLE_HOME under /opt/oracle
- Download 12.1.0.2 Grid/Database software – 21419221
- Download latest Opatch utility software – 6880880
- Download one-off patch required for your existing software version if any.
- Apply patches where required before upgrading proceeds
- Run Exachk before upgrade
- Navigate to grid software directory and run the Cluster verification utility as follows:
dm01db01-+ASM1 {/u01/app/oracle/software/grid}:ls -ltr
total 80
-rwxr-xr-x 1 oracle oinstall 500 Feb 6 2013 welcome.html
-rwxr-xr-x 1 oracle oinstall 5085 Dec 20 2013 runcluvfy.sh
-rwxr-xr-x 1 oracle oinstall 8534 Jul 7 2014 runInstaller
drwxr-xr-x 2 oracle oinstall 4096 Jul 7 2014 rpm
drwxrwxr-x 2 oracle oinstall 4096 Jul 7 2014 sshsetup
drwxrwxr-x 2 oracle oinstall 4096 Jul 7 2014 response
drwxr-xr-x 14 oracle oinstall 4096 Jul 7 2014 stage
-rwxr-xr-x 1 oracle oinstall 33934 Aug 7 2015 readme.html
drwxr-xr-x 4 oracle oinstall 4096 Jan 9 04:43 install
dm01db01-+ASM1 {/u01/app/oracle/software/grid}:./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/11.2.0.4/grid -dest_crshome /u01/app/12.1.0.2/grid -dest_version 12.1.0.2.0 -fixup -verbose
Performing pre-checks for cluster services setup
Checking node reachability…
Check: Node reachability from node “dm01db01”
Destination Node Reachable?
———————————— ————————
dm01db01 yes
dm01db02 yes
dm01db03 yes
dm01db04 yes
dm01db05 yes
dm01db06 yes
dm01db07 yes
dm01db08 yes
Result: Node reachability check passed from node “dm01db01”
Checking user equivalence…
Check: User equivalence for user “oracle”
Node Name Status
———————————— ————————
dm01db08 passed
dm01db07 passed
dm01db06 passed
dm01db05 passed
dm01db04 passed
dm01db03 passed
dm01db02 passed
dm01db01 passed
Result: User equivalence check passed for user “oracle”
Checking CRS user consistency
Result: CRS user consistency check successful
Checking network configuration consistency.
Result: Check for network configuration consistency passed.
Checking ASM disk size consistency
All ASM disks are correctly sized
Checking if ASM parameter file is in use by an ASM instance on the local node
Result: ASM instance is using parameter file “+SYSTEMDG/dm01-cluster/asmparameterfile/registry.253.726236669” on
node “dm01db01” on which upgrade is requested.
Checking OLR integrity…
Check of existence of OLR configuration file “/etc/oracle/olr.loc” passed
Check of attributes of OLR configuration file “/etc/oracle/olr.loc” passed
WARNING:
This check does not verify the integrity of the OLR contents. Execute ‘ocrcheck -local’ as a privileged user to verify the contents of OLR.
OLR integrity check passed
Checking node connectivity…
Checking hosts config file…
Node Name Status
———————————— ————————
dm01db01 passed
dm01db08 passed
dm01db07 passed
dm01db06 passed
dm01db05 passed
dm01db04 passed
dm01db03 passed
dm01db02 passed
Verification of the hosts config file successful
…………….
Check for Reverse path filter setting passed
Starting check for Network interface bonding status of private interconnect network interfaces …
Check for Network interface bonding status of private interconnect network interfaces passed
Starting check for /dev/shm mounted as temporary file system …
Check for /dev/shm mounted as temporary file system passed
Starting check for /boot mount …
Check for /boot mount passed
Starting check for zeroconf check …
Check for zeroconf check passed
Pre-check for cluster services setup was unsuccessful on all the nodes.
NOTE:
No fixable verification failures to fix
You can also redirect the output to the file and read it easily.
$ ./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/11.2.0.4/grid -dest_crshome /u01/app/12.1.0.2/grid -dest_version 12.1.0.2.0 -fixup -verbose > cluvfy_pre_upgrade.log
Pre-upgrade Information Tool
- Analyze your databases to be upgraded with the Pre-Upgrade Information Tool. Download and copy Pre-upgradeutility. Take a look at the MOS note below:
- Run the pre-upgrade utitlity as follows:
-rw-r–r– 1 oracle oinstall 101162 Jan 9 06:19 preupgrade_12.1.0.2.0_14_lf.zip
dm01db01-+ASM1 {/u01/app/oracle/software}:unzip preupgrade_12.1.0.2.0_14_lf.zip
Archive: preupgrade_12.1.0.2.0_14_lf.zip
inflating: preupgrd.sql
inflating: utluppkg.sql
dm01db01-+ASM1 {/u01/app/oracle/software}:. oraenv
ORACLE_SID = [+ASM1] ? orcldb1
The Oracle base remains unchanged with value /u01/app/oracle
dm01db01-orcldb1 {/u01/app/oracle/software}:sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Mon Jan 9 06:25:51 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> @preupgrd.sql
Loading Pre-Upgrade Package…
***************************************************************************
Executing Pre-Upgrade Checks in ORCLDB…
************************************************************
====>> ERRORS FOUND for ORCLDB <<====
The following are *** ERROR LEVEL CONDITIONS *** that must be addressed
prior to attempting your upgrade.
Failure to do so will result in a failed upgrade.
You MUST resolve the above errors prior to upgrade
************************************************************
====>> PRE-UPGRADE RESULTS for ORCLDB <<====
ACTIONS REQUIRED:
1. Review results of the pre-upgrade checks:
/u01/app/oracle/cfgtoollogs/orcldb/preupgrade/preupgrade.log
2. Execute in the SOURCE environment BEFORE upgrade:
/u01/app/oracle/cfgtoollogs/orcldb/preupgrade/preupgrade_fixups.sql
3. Execute in the NEW environment AFTER upgrade:
/u01/app/oracle/cfgtoollogs/orcldb/preupgrade/postupgrade_fixups.sql
***************************************************************************
Pre-Upgrade Checks in ORCLDB Completed.
***************************************************************************
***************************************************************************
***************************************************************************
- Review the preupgrade.log file for errors/issues:
- Run the preupgrade_fixups.sql produced by pre-upgrade utility above:
SQL*Plus: Release 11.2.0.4.0 Production on Mon Jan 9 06:29:50 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> @/u01/app/oracle/cfgtoollogs/orcldb/preupgrade/preupgrade_fixups.sql
Pre-Upgrade Fixup Script Generated on 2017-01-09 06:26:48 Version: 12.1.0.2 Build: 014
Beginning Pre-Upgrade Fixups…
Executing in container ORCLDB
**********************************************************************
Check Tag: DEFAULT_PROCESS_COUNT
Check Summary: Verify min process count is not too low
Fix Summary: Review and increase if needed, your PROCESSES value.
**********************************************************************
Fixup Returned Information:
WARNING: –> Process Count may be too low
Database has a maximum process count of 150 which is lower than the
default value of 300 for this release.
You should update your processes value prior to the upgrade
to a value of at least 300.
For example:
ALTER SYSTEM SET PROCESSES=300 SCOPE=SPFILE
or update your init.ora file.
**********************************************************************
**********************************************************************
Check Tag: EM_PRESENT
Check Summary: Check if Enterprise Manager is present
Fix Summary: Execute emremove.sql prior to upgrade.
**********************************************************************
Fixup Returned Information:
WARNING: –> Enterprise Manager Database Control repository found in the database
In Oracle Database 12c, Database Control is removed during
the upgrade. To save time during the Upgrade, this action
can be done prior to upgrading using the following steps after
copying rdbms/admin/emremove.sql from the new Oracle home
– Stop EM Database Control:
$> emctl stop dbconsole
– Connect to the Database using the SYS account AS SYSDBA:
SET ECHO ON;
SET SERVEROUTPUT ON;
@emremove.sql
Without the set echo and serveroutput commands you will not
be able to follow the progress of the script.
**********************************************************************
**********************************************************************
Check Tag: AMD_EXISTS
Check Summary: Check to see if AMD is present in the database
Fix Summary: Manually execute ORACLE_HOME/oraolap/admin/catnoamd.sql script to remove OLAP.
**********************************************************************
Fixup Returned Information:
INFORMATION: –> OLAP Catalog(AMD) exists in database
Starting with Oracle Database 12c, OLAP Catalog component is desupported.
If you are not using the OLAP Catalog component and want
to remove it, then execute the
ORACLE_HOME/olap/admin/catnoamd.sql script before or
after the upgrade.
**********************************************************************
**********************************************************************
Check Tag: APEX_UPGRADE_MSG
Check Summary: Check that APEX will need to be upgraded.
Fix Summary: Oracle Application Express can be manually upgraded prior to database upgrade.
**********************************************************************
Fixup Returned Information:
INFORMATION: –> Oracle Application Express (APEX) can be
manually upgraded prior to database upgrade
APEX is currently at version 3.2.1.00.12 and will need to be
upgraded to APEX version 4.2.5 in the new release.
Note 1: To reduce database upgrade time, APEX can be manually
upgraded outside of and prior to database upgrade.
Note 2: See MOS Note 1088970.1 for information on APEX
installation upgrades.
**********************************************************************
**********************************************************************
[Pre-Upgrade Recommendations]
**********************************************************************
*****************************************
********* Dictionary Statistics *********
*****************************************
Please gather dictionary statistics 24 hours prior to
upgrading the database.
To gather dictionary statistics execute the following command
while connected as SYSDBA:
EXECUTE dbms_stats.gather_dictionary_stats;
^^^ MANUAL ACTION SUGGESTED ^^^
**************************************************
************* Fixup Summary ************
4 fixup routines generated INFORMATIONAL messages that should be reviewed.
**************** Pre-Upgrade Fixup Script Complete *********************
PL/SQL procedure successfully completed.
- Take care of pre-upgrade Recommendation stated above:
- Increase processes parameter to 300
System altered.
- Drop OLAP Catalog
Synonym dropped.
Synonym dropped.
Synonym dropped.
Synonym dropped.
Synonym dropped.
Synonym dropped.
….
Type dropped.
Type dropped.
PL/SQL procedure successfully completed.
Role dropped.
PL/SQL procedure successfully completed.
1 row deleted.
- Gather dictionary statistics
PL/SQL procedure successfully completed.
Install and Upgrade Grid Infrastructure to 12.1.0.2
- Create the new Grid Infrastructure (GI_HOME) directory where 12.1.0.2 will be installed
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root chown oracle /u01/app/12.1.0.2/grid
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root chgrp -R oinstall /u01/app/12.1.0.2/grid
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root ls -l /u01/app/12.1.0.2/
dm01db01: total 4
dm01db01: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db02: total 4
dm01db02: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db03: total 4
dm01db03: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db04: total 4
dm01db04: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db05: total 4
dm01db05: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db06: total 4
dm01db06: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db07: total 4
dm01db07: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
dm01db08: total 4
dm01db08: drwxr-xr-x 2 oracle oinstall 4096 Jan 10 03:44 grid
- Unzip the software if not already done earlier.
dm01db01-orcldb1 {/u01/app/oracle/software}: unzip -q /u01/app/oracle/software/p21419221_121020_Linux-x86-64_6of10.zip -d /u01/app/oracle/software
- Unset the environment and run the installer
dm01db01- {/u01/app/oracle/software}:echo $ORACLE_HOME
dm01db01- {/u01/app/oracle/software}:echo $ORACLE_BASE
dm01db01- {/u01/app/oracle/software}:echo $ORACLE_SID
dm01db01- {/home/oracle}:export SRVM_USE_RACTRANS=true
dm01db01- {/home/oracle}:export DISPLAY=10.10.20.1:0.0
dm01db01- {/home/oracle}:cd /u01/app/oracle/software/grid/
dm01db01- {/u01/app/oracle/software/grid}:./runInstaller -J-Doracle.install.mgmtDB=false
Starting Oracle Universal Installer…
Checking Temp space: must be greater than 415 MB. Actual 7663 MB Passed
Checking swap space: must be greater than 150 MB. Actual 16378 MB Passed
Checking monitor: must be configured to display at least 256 colors
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2017-01-10_03-50-56AM. Please wait …dm01db01- {/u01/app/oracle/software/grid}:
Select Language and click Next
Select all the nodes you want to upgrade and click Next
Click Next
Select the OS group for OS authentication and click Next
Enter the Oracle Base and Grid Home and click Next
Click Next as we want to run the root script manually.
Prerequisites check in progress
Click Install to begin Installation process
Installation in progress
Follow the steps below before executing rootupgrade.sh script
- Update OPatch and when available apply the latest Bundle Patch on top of the 12.1.0.2 Grid Infrastructure Installation
dm01db01-orcldb1 {/u01/app/oracle/software}:dcli -g ~/dbs_group -l oracle unzip -oq -d /u01/app/12.1.0.2/grid /u01/app/oracle/software/p6880880_112000_Linux-x86-64.zip
dm01db01-orcldb1 {/u01/app/oracle/software}:dcli -g ~/dbs_group -l oracle /u01/app/12.1.0.2/grid/OPatch/opatch version
dm01db01: OPatch Version: 11.2.0.3.15
dm01db01:
dm01db01: OPatch succeeded.
dm01db02: OPatch Version: 11.2.0.3.15
dm01db02:
dm01db02: OPatch succeeded.
dm01db03: OPatch Version: 11.2.0.3.15
dm01db03:
dm01db03: OPatch succeeded.
dm01db04: OPatch Version: 11.2.0.3.15
dm01db04:
dm01db04: OPatch succeeded.
dm01db05: OPatch Version: 11.2.0.3.15
dm01db05:
dm01db05: OPatch succeeded.
dm01db06: OPatch Version: 11.2.0.3.15
dm01db06:
dm01db06: OPatch succeeded.
dm01db07: OPatch Version: 11.2.0.3.15
dm01db07:
dm01db07: OPatch succeeded.
dm01db08: OPatch Version: 11.2.0.3.15
dm01db08:
dm01db08: OPatch succeeded.
- When required relink the 12.1.0.2 Grid Infrastructure oracle binary with RDS
dm01db01: rds
dm01db02: rds
dm01db03: rds
dm01db04: rds
dm01db05: rds
dm01db06: rds
dm01db07: rds
dm01db08: rds
If the command does not return ‘rds’ relink as follows:
- Change SGA memory settings for ASM
SQL*Plus: Release 11.2.0.4.0 Production on Tue Jan 10 06:35:32 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 – 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> alter system set sga_max_size = 2G scope=spfile sid=’*’;
System altered.
SQL> alter system set sga_target = 2G scope=spfile sid=’*’;
System altered.
- Verify values for memory_target, memory_max_target and use_large_pages
SQL> col name format a25
SQL> col value format a20
SQL> set lines 200
SQL> set pages 200
SQL> select sid, name, value from v$spparameter where name in(‘memory_target’,’memory_max_target’,’use_large_pages’);
SID NAME VALUE
—- ————————- ——————–
* use_large_pages TRUE
* memory_target 0
* memory_max_target
- Verify no active rebalance is running
COUNT(*)
———-
0
- Now Execute rootupgrade.sh on each database server
[root@dm01db01 ~]# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10
[root@dm01db01 ~]# /u01/app/12.1.0.2/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/12.1.0.2/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file “dbhome” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin …
The file “oraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin …
The file “coraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to /usr/local/bin …
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0.2/grid/crs/install/crsconfig_params
2017/01/10 06:52:50 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA)
Collector.
2017/01/10 06:52:51 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2017/01/10 06:52:53 CLSRSC-464: Starting retrieval of the cluster configuration data
2017/01/10 06:53:02 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2017/01/10 06:53:02 CLSRSC-363: User ignored prerequisites during installation
2017/01/10 06:53:13 CLSRSC-515: Starting OCR manual backup.
2017/01/10 06:53:15 CLSRSC-516: OCR manual backup successful.
2017/01/10 06:53:19 CLSRSC-468: Setting Oracle Clusterware and ASM to rolling migration mode
2017/01/10 06:53:19 CLSRSC-482: Running command: ‘/u01/app/12.1.0.2/grid/bin/asmca -silent -upgradeNodeASM –
nonRolling false -oldCRSHome /u01/app/11.2.0.4/grid -oldCRSVersion 11.2.0.4.0 -nodeNumber 1 -firstNode true -startRolling true’
ASM configuration upgraded in local node successfully.
2017/01/10 06:53:27 CLSRSC-469: Successfully set Oracle Clusterware and ASM to rolling migration mode
2017/01/10 06:53:27 CLSRSC-466: Starting shutdown of the current Oracle Grid Infrastructure stack
2017/01/10 06:53:53 CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
OLR initialization – successful
2017/01/10 06:58:48 CLSRSC-329: Replacing Clusterware entries in file ‘oracle-ohasd.conf’
CRS-4133: Oracle High Availability Services has been stopped.
kgfnGetFacility: facility=0x1564768
kgfnInitDiag: diagctx=0x13e4160
CRS-4123: Oracle High Availability Services has been started.
2017/01/10 07:03:18 CLSRSC-472: Attempting to export the OCR
2017/01/10 07:03:18 CLSRSC-482: Running command: ‘ocrconfig -upgrade oracle oinstall’
2017/01/10 07:03:29 CLSRSC-473: Successfully exported the OCR
2017/01/10 07:03:33 CLSRSC-486:
At this stage of upgrade, the OCR has changed.
Any attempt to downgrade the cluster after this point will require a complete cluster outage to restore the OCR.
2017/01/10 07:03:33 CLSRSC-541:
To downgrade the cluster:
1. All nodes that have been upgraded must be downgraded.
2017/01/10 07:03:33 CLSRSC-542:
2. Before downgrading the last node, the Grid Infrastructure stack on all other cluster nodes must be down.
2017/01/10 07:03:33 CLSRSC-543:
3. The downgrade command must be run on the node dm01db04 with the ‘-lastnode’ option to restore global configuration data.
2017/01/10 07:04:04 CLSRSC-343: Successfully started Oracle Clusterware stack
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 11g Release 2.
Successfully taken the backup of node specific configuration in OCR.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
2017/01/10 07:04:13 CLSRSC-474: Initiating upgrade of resource types
2017/01/10 07:04:38 CLSRSC-482: Running command: ‘upgrade model -s 11.2.0.4.0 -d 12.1.0.2.0 -p first’
2017/01/10 07:04:38 CLSRSC-475: Upgrade of resource types successfully initiated.
2017/01/10 07:04:43 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster … succeeded
Go back to the installation screen and click Ok
dm01db01-+ASM1 {/home/oracle}:dcli -g dbs_group -l oracle ‘/u01/app/12.1.0.2/grid/bin/crsctl query crs activeversion’
dm01db01: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db02: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db03: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db04: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db05: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db06: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db07: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db08: Oracle Clusterware active version on the cluster is [12.1.0.2.0]
dm01db01-+ASM1 {/home/oracle}:dcli -g dbs_group -l oracle ‘/u01/app/12.1.0.2/grid/bin/crsctl query crs softwareversion’
dm01db01: Oracle Clusterware version on node [dm01db01] is [12.1.0.2.0]
dm01db02: Oracle Clusterware version on node [dm01db02] is [12.1.0.2.0]
dm01db03: Oracle Clusterware version on node [dm01db03] is [12.1.0.2.0]
dm01db04: Oracle Clusterware version on node [dm01db04] is [12.1.0.2.0]
dm01db05: Oracle Clusterware version on node [dm01db05] is [12.1.0.2.0]
dm01db06: Oracle Clusterware version on node [dm01db06] is [12.1.0.2.0]
dm01db07: Oracle Clusterware version on node [dm01db07] is [12.1.0.2.0]
dm01db08: Oracle Clusterware version on node [dm01db08] is [12.1.0.2.0]
dm01db01-+ASM1 {/home/oracle}:/u01/app/12.1.0.2/grid/bin/crsctl check cluster -all
**************************************************************
dm01db01:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db02:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db03:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db04:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db05:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db06:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db07:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db08:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
dm01db01-+ASM1 {/home/oracle}:vi .bash_profile
dm01db01-+ASM1 {/home/oracle}:vi .exadata_profile
SQL> set lines 200
SQL> set pages 200
SQL> select * from v$version;
BANNER CON_ID
——————————————————————————– ———-
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 – 64bit Production 0
PL/SQL Release 12.1.0.2.0 – Production 0
CORE 12.1.0.2.0 Production 0
TNS for Linux: Version 12.1.0.2.0 – Production 0
NLSRTL Version 12.1.0.2.0 – Production 0
ASMCMD> lsct -g DATA
Instance_ID DB_Name Status Software_Version Compatible_version Instance_Name Disk_Group
1 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM1 DATA
2 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM2 DATA
3 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM3 DATA
4 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM4 DATA
5 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM5 DATA
6 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM6 DATA
7 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM7 DATA
8 +ASM CONNECTED 12.1.0.2.0 12.1.0.2.0 +ASM8 DATA
1 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb1 DATA
2 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb2 DATA
3 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb3 DATA
4 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb4 DATA
5 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb5 DATA
6 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb6 DATA
7 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb7 DATA
8 orcldb CONNECTED 11.2.0.4.0 11.2.0.4.0 orcldb8 DATA
[root@dm01db01 ~]# /u01/app/12.1.0.2/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 409568
Used space (kbytes) : 10592
Available space (kbytes) : 398976
ID : 1119121028
Device/File Name : +SYSTEMDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
dm01db01-+ASM1 {/home/oracle}:crsctl query css votedisk
## STATE File Universal Id File Name Disk group
— —– —————– ——— ———
1. ONLINE 6938ea097e324f0fbf48da02892564cb (o/192.168.1.9/SYSTEMDG_CD_02_dm01cel01) [SYSTEMDG]
2. ONLINE 42561e3fd68a4f7dbfe0687fb6bb6a1f (o/192.168.1.22/SYSTEMDG_CD_08_dm01cel14) [SYSTEMDG]
3. ONLINE 2cc2afc24d504f99bfb05a60620e01ef (o/192.168.1.13/SYSTEMDG_CD_02_dm01cel05) [SYSTEMDG]
Located 3 voting disk(s).
Conclusion
In this article we have successfully upgraded Oracle Grid Infrastructure from 11.2.0.4 to 12.1.0.2. Oracle 12.1.0.2 includes the support for Smart Scan, Storage Indexes and IORM, so Customer can take full advantage of running 12.1.0.2 on Exadata Database Machine. The upgrade from 11.2.0.4 to 12.1.0.2 works seamless if all the prerequisites are met properly.