Home
 | 
Articles
 | 
Installation
 | 
Oracle Database 12c Release 2 (12.2) RAC On Oracle Linux 7
 | 
Oracle Database 12c Release 2 (12.2) RAC On Oracle Linux 7
 
Last modified: June 2017
All Archives
Topics
Comments
 
           
Pause Play
Let's save before they become  EXTINCT
Let's save  VEGETATION
Let's save the god  NATURE
Thanks for saving the  BLUE PLANET!
Overview
»
  
This article describes the installation of Oracle Grid Infrastructure and Cluster Database 12c Release 2 (2 nodes) on Oracle Linux 7.3 using VirtualBox.
In order to save time, I will build the first guest machine (node1 which I would call it as olinux7301.shannura.com) and install the prerequisite packages, create users, groups and installation paths etc. and then clone it to create another node (olinux7302.shannura.com).
Environment overview

Operating System: Oracle Linux 7.3
Kernel: 4.1.12-61.1.18.el7uek.x86_64
Hostname(s): olinux7301, olinux7302
Virtual Hostname(s): olinux7301-vip, olinux7302-vip
Public Interface(s): 172.168.190.101, 172.168.190.102 
Private Interface(s): 192.168.190.101, 192.168.190.102
Virtual Interface(s): 172.168.190.201, 172.168.190.202
Cluster: shannuracluster
SCAN: shannuracluster-scan
SCAN-VIP(s): 172.168.190.211, 172.168.190.212, 172.168.190.213
Softwares Download

linuxx64_12201_grid_home.zip
linuxx64_12201_database.zip
Installing the Preinstall packages
$
$ yum install oracle-database-server-12cR2-preinstall -y
Additional packages
In addition, it is worth installing these packages manually although most of them may have already been installed if you executed the preinstall package.
$<>  
$ yum install binutils -y
$ yum install compat-libstdc++-33 -y
$ yum install compat-libstdc++-33.i686 -y
$ yum install gcc -y
$ yum install gcc-c++ -y
$ yum install glibc -y
$ yum install glibc.i686 -y
$ yum install glibc-devel -y
$ yum install glibc-devel.i686 -y
$ yum install ksh -y
$ yum install libgcc -y
$ yum install libgcc.i686 -y
$ yum install libstdc++ -y
$ yum install libstdc++.i686 -y
$ yum install libstdc++-devel -y
$ yum install libstdc++-devel.i686 -y
$ yum install libaio -y
$ yum install libaio.i686 -y
$ yum install libaio-devel -y
$ yum install libaio-devel.i686 -y
$ yum install libXext -y
$ yum install libXext.i686 -y
$ yum install libXtst -y
$ yum install libXtst.i686 -y
$ yum install libX11 -y
$ yum install libX11.i686 -y
$ yum install libXau -y
$ yum install libXau.i686 -y
$ yum install libxcb -y
$ yum install libxcb.i686 -y
$ yum install libXi -y
$ yum install libXi.i686 -y
$ yum install make -y
$ yum install sysstat -y
$ yum install unixODBC -y
$ yum install unixODBC-devel -y
$ yum install zlib-devel -y
$ yum install zlib-devel.i686 -y	
Resource limits

/etc/security/limits.d/oracle-database-server-12cR2-preinstall.conf
/etc/limits.d/limits.conf
$
oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    16384
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768
oracle   hard   memlock    134217728
oracle   soft   memlock    134217728
grid     soft   nofile    1024
grid     hard   nofile    65536
grid     soft   nproc    16384
grid     hard   nproc    16384
grid     soft   stack    10240
grid     hard   stack    32768
grid     hard   memlock    134217728
grid     soft   memlock    134217728	
Users, Groups & Permissions
$
$ groupadd -g 54323 oper
$ groupadd -g 54327 asmdba
$ groupadd -g 54328 asmoper
$ groupadd -g 54329 asmadmin

$ useradd -u 54322 -g oinstall -G dba,asmdba,asmoper,asmadmin grid
$ usermod -a -G asmdba,oper oracle

$ passwd oracle
$ passwd grid
/etc/hosts

127.0.0.1         localhost.localdomain localhost

172.168.190.101   olinux7301.shannura.com olinux7301
172.168.190.102   olinux7302.shannura.com olinux7302

172.168.190.201   olinux7301-vip.shannura.com olinux7301-vip
172.168.190.202   olinux7302-vip.shannura.com olinux7302-vip

#172.168.190.211   shannuracluster-scan.shannura.com shannuracluster-scan
#172.168.190.212   shannuracluster-scan.shannura.com shannuracluster-scan
#172.168.190.213   shannuracluster-scan.shannura.com shannuracluster-scan

192.168.190.101   olinux7301-priv.shannura.com olinux7301-priv
192.168.190.102   olinux7302-priv.shannura.com olinux7302-priv
DNS and SCAN name
Below machine is my third virtual machine which is the DNS server to resolve the participating nodes' hostnames as well as SCAN name(s). Follow this separate post on how to setup a DNS server.
DNS Server:	Hostname: rhel54.shannura.com
		Public Interface: 172.168.190.200
		Gateway: 172.168.190.100
				
		SCAN NAME: shannuracluster-scan.shannura.com
		IP Address:	172.168.190.211
				172.168.190.212
				172.168.190.213
My DNS server looks perfect folloiwng a quick nslookup check. The DNS server returns the correct IP address for the hosts I have set it up
$<>  
$ nslookup olinux7301.shannura.com      (should return 172.168.190.101)
$ nslookup olinux7302.shannura.com      (should return 172.168.190.102)
$ nslookup olinux7301-vip.shannura.com  (should return 172.168.190.201)
$ nslookup olinux7302-vip.shannura.com  (should return 172.168.190.202)
$ nslookup shannuracluster-scan         (should return 172.168.190.211, 172.168.190.212, 172.168.190.213)
Shared Memory
$
$ mount -o remount,size=4G /dev/shm
Add this line in /etc/fstab

none      /dev/shm        tmpfs   defaults,size=4G        0 0
SELinux flag

/etc/selinux/config

SELINUX=permissive
Firewall
$
$ systemctl stop firewalld
$ systemctl disable firewalld
NTP (Chrony on OL7/RHEL7)
$
$ systemctl enable chronyd.service
$ systemctl start chronyd.service
Directory structure(s)
$
$ mkdir -p /u01/app/12.2.0.1/grid
$ mkdir -p /u01/app/grid
$ mkdir -p /u01/app/oracle/product/12.2.0.1/db_1
$ chown -R grid:oinstall /u01
$ chown -R oracle:oinstall /u01/app/oracle
$ chmod -R 775 /u01
Environment variables

/home/grid/.bash_profile
$
$ cat /home/grid/.bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi  
ORACLE_SID=+ASM; export ORACLE_SID
JAVA_HOME=/usr/bin/java; export JAVA_HOME
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/12.2.0.1/grid; export ORACLE_HOME
ORACLE_TERM=xterm; export ORACLE_TERM
TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/local/bin
export PATH
export TEMP=/tmp
export TMPDIR=/tmp
umask 022

/home/oracle/.bash_profile
$
$ cat /home/oracle/.bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi  
ORACLE_SID=oradb; export ORACLE_SID
ORACLE_UNQNAME=oradb; export ORACLE_UNQNAME
JAVA_HOME=/usr/bin/java; export JAVA_HOME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/12.2.0.1/db_1; 
export ORACLE_HOME
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"
export NLS_DATE_FORMAT
TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/local/bin
export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib 
export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export CLASSPATH
THREADS_FLAG=native; export THREADS_FLAG
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
Clone the Virtual Machine
It's time to shutdown node1 and clone this guest and make necessary changes (e.g. hostname, network interfaces (public & private)) in the cloned node to ensure they can participate in the cluster.

Hostname(s): olinux7301, olinux7302
Public Interface(s): 172.168.190.101, 172.168.190.102 
Private Interface(s): 192.168.190.101, 192.168.190.102
Sharable Virtual Disks
I have used Linux's UDEV instead of ASMLib. Earlier I had created 4 sharable virtual disks and configured UDEV rules (see how to use UDEV rules).
$
$ ls -al /dev/oracleasm/*
lrwxrwxrwx. 1 root root 7 Jun  6 19:09 /dev/oracleasm/asm-disk1 -> ../sdb1
lrwxrwxrwx. 1 root root 7 Jun  6 19:09 /dev/oracleasm/asm-disk2 -> ../sdc1
lrwxrwxrwx. 1 root root 7 Jun  6 19:09 /dev/oracleasm/asm-disk3 -> ../sdd1
lrwxrwxrwx. 1 root root 7 Jun  6 19:09 /dev/oracleasm/asm-disk4 -> ../sde1
Ensure, these shared disks are accessible from both nodes.
Grid Infrastructure Installation
$
(as grid user - on node1) 
$ export SOFT_LOC=/u02/stage
$ cd /u01/app/12.2.0.1/grid
$ unzip -q $SOFT_LOC/linuxx64_12201_grid_home.zip
$
(as root user on - node1)
$ cd /u01/app/12.2.0.1/grid/cv/rpm
$ rpm -Uvh cvuqdisk*
$ scp ./cvuqdisk* root@172.168.190.102:/tmp
$ ssh root@172.168.190.102 rpm -Uvh /tmp/cvuqdisk*
$ exit
$
(back as grid user on node1)
$ cd /u01/app/12.2.0.1/grid
$ ./gridSetup.sh
The remaining steps are self explanatory from the screenshots.
I got these errors (PRVG-13606) towards the end, they are all related to time synchronization which I can ignore in this environment and proceed further.

Verifying chrony daemon is synchronized with at least one external time source ...FAILED (PRVG-13606)
Verifying Network Time Protocol (NTP) ...FAILED (PRVG-13606)
Verifying Clock Synchronization ...FAILED (PRVG-13606)
The grid infrastructure installation is now complete.
$<>  
[grid@olinux7301 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.DATA.dg
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.FRA.dg
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.chad
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.net1.network
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.ons
               ONLINE  ONLINE       olinux7301               STABLE
               ONLINE  ONLINE       olinux7302               STABLE
ora.proxy_advm
               OFFLINE OFFLINE      olinux7301               STABLE
               OFFLINE OFFLINE      olinux7302               STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       olinux7302               STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       olinux7301               STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       olinux7301               STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       olinux7301               192.168.190.101,STABLE
ora.asm
      1        ONLINE  ONLINE       olinux7301               Started,STABLE
      2        ONLINE  ONLINE       olinux7302               Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       olinux7301               STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       olinux7301               Open,STABLE
ora.olinux7301.vip
      1        ONLINE  ONLINE       olinux7301               STABLE
ora.olinux7302.vip
      1        ONLINE  ONLINE       olinux7302               STABLE
ora.qosmserver
      1        ONLINE  ONLINE       olinux7301               STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       olinux7302               STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       olinux7301               STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       olinux7301               STABLE
--------------------------------------------------------------------------------
Database Software Installation
Unzip the database software (as oracle user) and execute './runInstaller'.
$
$ cd /u02/stage
$ unzip linuxx64_12201_database.zip
$ cd database
$ ./runInstaller
The remaining steps are self explanatory from the screenshots.
Happy to ignore all of these in this environment. They are related to time synchronization and DNS.
Installation of Oracle Database software is now complete.
Create Database (phoenix)
Before launching 'dbca', as 'grid' user launch 'asmca' and create the two additional DISK groups (PHX & FRA) from the available candidate disks as shown here.
$
[grid@olinux7301 ~]$ asmca
$
[oracle@olinux7301 ~]$ dbca
The remaining steps are self explanatory from the screenshots.
$<>  
[oracle@olinux7301 ~]$ srvctl config database -database phoenix
Database unique name: phoenix
Database name: phoenix
Oracle home: /u01/app/oracle/product/12.2.0.1/db_1
Oracle user: oracle
Spfile: +PHX/PHOENIX/PARAMETERFILE/spfile.268.946157843
Password file: +PHX/PHOENIX/PASSWORD/pwdphoenix.256.946157137
Domain: shannura.com
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: FRA,PHX
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: dba
OSOPER group: oper
Database instances: phoenix1,phoenix2
Configured nodes: olinux7301,olinux7302
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed
Please write your comment if this article was useful.

ShanNura

/
You might want to read this:
Dataguard Switchover in 12c