Install 11gR2 Oracle Restart (Grid Infrastructure for Standalone) on Centos 5.5 with role seperation

Configuration

Server: Intel Core i5, 8GB RAM
OS: Centos 5.5 (64 bit)
Grid Infrastructure owner: grid
Oracle Database software owner: oracle

1. Prepare the system

    1.1 Install the packages 

    yum -y install binutils-2.*
    yum -y install compat-libstdc++-33*
    yum -y install elfutils-libelf-0.*
    yum -y install elfutils-libelf-devel-*
    yum -y install gcc-4.*
    yum -y install gcc-c++-4.*
    yum -y install glibc-2.*
    yum -y install glibc-common-2.*
    yum -y install glibc-devel-2.*
    yum -y install glibc-headers-2.*
    yum -y install ksh-2*
    yum -y install libaio-0.*
    yum -y install libaio-devel-0.*
    yum -y install libgcc-4.*
    yum -y install libstdc++-4.*
    yum -y install libstdc++-devel-4.*
    yum -y install make-3.*
    yum -y install sysstat-7.*
    yum -y install unixODBC-2.*
    yum -y install unixODBC-devel-2.*
    yum -y install pdksh*

    1.2 Create required users and groups 

    Groups:

    [root@oldb ~]# groupadd -g 1501 oinstall
    [root@oldb ~]# groupadd -g 1502 dba
    [root@oldb ~]# groupadd -g 1503 oper


    [root@oldb 11.2]# groupadd -g 1504 asmadmin
    [root@oldb 11.2]# groupadd -g 1505 asmdba
    [root@oldb 11.2]# groupadd -g 1506 asmoper

    Create Oracle user:

    [root@oldb ~]# useradd -u 1501 -m -g oinstall -G dba,oper oracle
    [root@oldb ~]# id oracle
    uid=1501(oracle) gid=1501(oinstall) groups=1501(oinstall),1502(dba),1503(oper)

    Add oracle to asmdba group

    [root@oldb 11.2]# usermod -a -G asmdba oracle

    #Thought to change the uid for oracle to 1502 and create user grid with 1501

    [root@oldb 11.2]# usermod -u 1502 oracle

    [root@oldb 11.2]# id oracle
    uid=1502(oracle) gid=1501(oinstall) groups=1501(oinstall),1502(dba),1503(oper),1505(asmdba)

    Note: Per oracle documentation, for Oracle restart “For Oracle grid infrastructure only, the grid user (grid) needs to be in the OSDBA group of every database home” .
    Otherwise, database creation will fail with “insufficient privileges to start resource”.

    Create user “grid” including oracle DBA group “dba” also as secondary group of grid user.

    [root@oldb 11.2]# useradd -u 1501 -g oinstall -G asmadmin,asmdba,asmoper,dba grid

    [root@oldb ~]# id grid
    uid=1501(grid) gid=1501(oinstall) groups=1501(oinstall),1502(dba),1504(asmadmin),1505(asmdba),1506(asmoper)

    # Change password of “grid” and “oracle” user using

    $ passwd grid
    $ passwd oracle

    1.3 Kernel parameters  

    /etc/sysctl.conf – Configure below kernel parameter values

    kernel.shmmni = 4096
    kernel.sem = 250 32000 100 128
    fs.file-max = 6815744
    fs.aio-max-nr = 1048576
    net.ipv4.ip_local_port_range = 9000 65500
    net.core.rmem_default = 262144
    net.core.rmem_max = 4194304
    net.core.wmem_default = 262144
    net.core.wmem_max = 1048576

    #reload latest values using

    /sbin/sysctl -p

    1.4 Set Shell Limits

    /etc/security/limits.conf - set the shell limits
    grid soft nproc 2047
    grid hard nproc 16384
    grid soft nofile 1024
    grid hard nofile 65536
    oracle soft nproc 2047
    oracle hard nproc 16384
    oracle soft nofile 1024
    oracle hard nofile 65536
    1.5 Session Limits 

    /etc/pam.d/login
    session required pam_limits.so

    1.6 Profile changes for “grid” and “oracle” 

    /etc/profile – make required profile settings

    if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
    if [ $SHELL = "/bin/ksh" ]; then
    ulimit -p 16384
    ulimit -n 65536
    else
    ulimit -u 16384 -n 65536
    fi
    umask 022
    fi

    1.7 Create the directories for installation 

    For orainventory

    # mkdir -p /u01/app/oraInventory
    # chown -R grid:oinstall /u01/app/oraInventory
    # chmod -R 775 /u01/app/oraInventory

    For grid install

    # mkdir -p /u01/11.2.0/grid
    # chown -R grid:oinstall /u01/11.2.0/grid
    # chmod -R 775 /u01/11.2.0/grid

    For database install

    # mkdir -p /u01/app/oracle
    # mkdir /u01/app/oracle/cfgtoollogs --needed to ensure that dbca is able to run after the rdbms installation.
    # chown -R oracle:oinstall /u01/app/oracle
    # chmod -R 775 /u01/app/oracle


    # mkdir -p /u01/app/oracle/product/11.2.0/db_1
    # chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/db_1
    # chmod -R 775 /u01/app/oracle/product/11.2.0/db_1

2. ASM Library Configuration

    2.1 Install asmlib 

    Download the respective asm library files from OTN

    ASMLib parent page
    http://www.oracle.com/technetwork/topics/linux/index-101839.html

    All ASMLib installations require appropriate oracleasmlib and oracleasm-support library and tools packages along with the driver package. For Red-hat Linux they can be downloaded from

    http://www.oracle.com/technetwork/topics/linux/downloads/rhel5-084877.html

    Download:
    oracleasm-support-2.1.3-1.el5.x86_64.rpm
    oracleasmlib-2.0.4-1.el5.x86_64.rpm
    oracleasm-2.6.18-194.8.1.el5-2.0.5-1.el5.x86_64.rpm

    Then

    [root@oldb Downloads]# rpm -ivh oracleasm-support-2.1.3-1.el5.x86_64.rpm \
    > oracleasmlib-2.0.4-1.el5.x86_64.rpm \
    > oracleasm-2.6.18-194.8.1.el5-2.0.5-1.el5.x86_64.rpm
    warning: oracleasm-support-2.1.3-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
    Preparing... ########################################### [100%]
    1:oracleasm-support ########################################### [ 33%]
    2:oracleasm-2.6.18-194.8.########################################### [ 67%]
    3:oracleasmlib ########################################### [100%]
    [root@oldb Downloads]#
    [root@oldb Downloads]#
    [root@oldb Downloads]#

    2.2 Configure the driver

    [root@oldb Downloads]# /etc/init.d/oracleasm configure
    Configuring the Oracle ASM library driver.
     

    This will configure the on-boot properties of the Oracle ASM library
    driver. The following questions will determine whether the driver is
    loaded on boot and what permissions it will have. The current values
    will be shown in brackets (‘[]’). Hitting without typing an
    answer will keep that current value. Ctrl-C will abort.

    Default user to own the driver interface []: grid
    Default group to own the driver interface []: asmdba
    Start Oracle ASM library driver on boot (y/n) [n]: y
    Scan for Oracle ASM disks on boot (y/n) [y]: y
    Writing Oracle ASM library driver configuration: done
    Initializing the Oracle ASMLib driver: [ OK ]
    Scanning the system for Oracle ASMLib disks: [ OK ]
    [root@oldb Downloads]#

    2.2 Create the ASM disks  

    Note: RAW disks are required for creating disks using asmlib.

    [root@oldb Downloads]# oracleasm createdisk ASM1 /dev/sda6
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm listdisks
    ASM1
    [root@oldb Downloads]# oracleasm createdisk ASM2 /dev/sda7
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm createdisk ASM3 /dev/sda8
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm createdisk ASM3 /dev/sda9
    Disk "ASM3" already exists
    [root@oldb Downloads]# oracleasm createdisk ASM4 /dev/sda9
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm createdisk ASM5 /dev/sda10
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm createdisk ASM6 /dev/sda11
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm createdisk ASM7 /dev/sda12
    Writing disk header: done
    Instantiating disk: done
    [root@oldb Downloads]# oracleasm createdisk ASM8 /dev/sda13
    Unable to query file "/dev/sda13": No such file or directory
    [root@oldb Downloads]#
    [root@oldb Downloads]#

    List and check the permissions of asm disks

    [root@oldb Downloads]# oracleasm listdisks
    ASM1
    ASM2
    ASM3
    ASM4
    ASM5
    ASM6
    ASM7

    [root@oldb Downloads]# ls -ltr /dev/oracleasm/disks
    total 0
    brw-rw—- 1 grid asmdba 8, 6 Sep 7 19:53 ASM1
    brw-rw—- 1 grid asmdba 8, 7 Sep 7 19:53 ASM2
    brw-rw—- 1 grid asmdba 8, 8 Sep 7 19:53 ASM3
    brw-rw—- 1 grid asmdba 8, 9 Sep 7 19:54 ASM4
    brw-rw—- 1 grid asmdba 8, 10 Sep 7 19:54 ASM5
    brw-rw—- 1 grid asmdba 8, 11 Sep 7 19:54 ASM6
    brw-rw—- 1 grid asmdba 8, 12 Sep 7 19:54 ASM7
    [root@oldb Downloads]#

3. Grid Infrastructure installation

Centos is unsupported, so to prevent the message “ADVM/ACFS is not supported on centos-release-5-5.el5.centos” while running root.sh , do

$echo "redhat-release-5Server-5" > /tmp/.linux_release

To prevent deletion or modification of .linux_release file

/usr/bin/chattr +i /tmp/.linux_release

(well i did not set this prior to starting the installation, and the message can be seen in the o/p of root.sh)

    3.1 Install Clusterware for Standalone  

    As user “grid” start the installation.
    Note: The only supported method to change the install id after installation is to reinstall the whole thing.


    $ su - grid
    $ ./runInstaller

    Choosing the options as below:

    Select the diskgroup
    Install and configure Grid Infrastructure for a Standalone Server
    ASM DBA admin (OSDBA) Group - asmdba
    ASM Instance Administration Operator (OSOPER) - asmoper
    ASM Instance Administrator (OSASM) Group - asmadmin
    Oracle Base = /u01/app/oracle
    Software Location = /u01/11.2.0/grid
    Check the setting one final time and vrooomm....

    3.2 orainstRoot.sh and root.sh for standalone clusterware 

    Run orainstRoot.sh and root.sh as root when prompted

    [root@oldb ~]# cd /u01/app/oraInventory/
    [root@oldb oraInventory]# ls -ltr
    total 28
    drwxrwx--- 2 grid oinstall 4096 Sep 7 20:20 logs
    -rw-rw---- 1 grid oinstall 293 Sep 7 20:20 oraInstaller.properties
    drwxrwx--- 2 grid oinstall 4096 Sep 7 20:20 oui
    -rw-rw---- 1 grid oinstall 37 Sep 7 20:22 install.platform
    drwxrwx--- 2 grid oinstall 4096 Sep 7 20:22 ContentsXML
    -rwxrwx--- 1 grid oinstall 1623 Sep 7 20:22 orainstRoot.sh
    -rw-rw---- 1 grid oinstall 56 Sep 7 20:22 oraInst.loc

    [root@oldb oraInventory]# ./orainstRoot.sh
    Changing permissions of /u01/app/oraInventory.
    Adding read,write permissions for group.
    Removing read,write,execute permissions for world.

    Changing groupname of /u01/app/oraInventory to oinstall.
    The execution of the script is complete.
    [root@oldb oraInventory]#
    [root@oldb oraInventory]#
    [root@oldb oraInventory]# cd /u01/11.*/grid
    [root@oldb grid]# ls -ltr | grep root
    -rwxr-x— 1 grid oinstall 652 Sep 7 20:21 root.sh
    [root@oldb grid]# ./root.sh
    Running Oracle 11g root.sh script…

    The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME= /u01/11.2.0/grid

    Enter the full pathname of the local bin directory: [/usr/local/bin]:
    Copying dbhome to /usr/local/bin …
    Copying oraenv to /usr/local/bin …
    Copying coraenv to /usr/local/bin …

    Creating /etc/oratab file…
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-09-07 20:25:37: Checking for super user privileges
    2010-09-07 20:25:37: User has super user privileges
    2010-09-07 20:25:37: Parsing the host name
    Using configuration parameter file: /u01/11.2.0/grid/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user ‘grid’, privgrp ‘oinstall’..
    Operation successful.
    CRS-4664: Node oldb successfully pinned.
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    ADVM/ACFS is not supported on centos-release-5-5.el5.centos

    oldb 2010/09/07 20:26:07 /u01/11.2.0/grid/cdata/oldb/backup_20100907_202607.olr
    Successfully configured Oracle Grid Infrastructure for a Standalone Server
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer…

    Checking swap space: must be greater than 500 MB. Actual 7999 MB Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /u01/app/oraInventory
    ‘UpdateNodeList’ was successful.
    [root@oldb grid]#

     

    3.3 Install Database software  

    As “oracle” user

    $ su - oracle
    $ cd ~/database
    $ ./runInstaller

    Choosing the options as below:

    Install database software only
    single instance database installation
    Enterprise edition
    Oracle Base: /u01/app/oracle Software location: /u01/app/oracle/product/11.2.0/db_1
    OSDBA : dba OSOPER:oper
    Final verification
    vrooommm...

    3.4 root.sh for database binaries  

    Run root.sh when prompted

    [root@oldb db_1]# ./root.sh
    Running Oracle 11g root.sh script...

    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1

    Enter the full pathname of the local bin directory: [/usr/local/bin]:
    The file “dbhome” already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    The file “oraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    The file “coraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:

    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    Finished product-specific root actions.
    [root@oldb db_1]#

4. ASMCA and additional diskgroups

    ASMCA has to be run as the owner of grid installation, which here is “grid”. You cannot run that as root.

    $ su - grid
     

    $ cd /u01/11.2.0/grid/bin

    $ ./asmca

    Below pictures shows that I am creating two more diskgroups with external and high redundancy.

5. Create the Database

    This is the final step; creating the database using dbca. 


    $ su - oracle

    [oracle@oldb ~]$ export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
    [oracle@oldb ~]$ export PATH=$ORACLE_HOME/bin:$PATH:

    [oracle@oldb ~]$ dbca

Note: You should be able to create the database as you normally would using the ASM disks without any issues. If you are chose to configure Database Control, take a note of the repository encryption key file displayed at the very end before exiting. ( ex:- /u01/app/oracle/product/11.2.0/db_1/oldb.localdomain_mdiuat/sysman/config/emkey.ora)

Advertisements
Comments
  1. Lucy zhang says:

    Hi, do I have to have user grid? Can I use user oracle for both ASM and oracle database?
    Thanks,
    Lucy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s