Netsoftmate

Tag: Shutdown Exadata Database Machine

  • Steps to Shutdown Exadata Database Machine

    Overview
    When you perform power maintenance, data center maintenance or moving servers from one data center to another you need to shutdown the Clusterware/Database/Servers. If you have Exadata Database Machines sitting in your data center then you need to shutdown it as well. The process of shutting down Exadata Database machine is slightly different when compared to servers or cluster. We should follow a sequence of steps to shutdown Exadata components. 


    In this article, we will demonstrate the steps to shutdown Exadata Database Machine to carry out maintenance task.


    Environment
    Exadata X5-2 Full Rack


    • Log in to the first Exadata compute node as as root user.
    • Change to root home directory and create the following files

    [root@dm01db01 ~]# cd /root


    [root@dm01db01 ~]# vi dbs_group
    dm01db01
    dm01db02
    dm01db03
    dm01db04
    dm01db05
    dm01db06
    dm01db07
    dm01db08


    [root@dm01db01 ~]# vi cell_group
    dm01cel01
    dm01cel02
    dm01cel03
    dm01cel04
    dm01cel05
    dm01cel06
    dm01cel07
    dm01cel08
    dm01cel09
    dm01cel10
    dm01cel11
    dm01cel12
    dm01cel13
    dm01cel14

    • Check if Grid Infrastructure is enabled for autostart:

    [root@dm01db01 ~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl config crs
    dm01db01: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db02: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db03: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db04: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db05: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db06: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db07: CRS-4622: Oracle High Availability Services autostart is enabled.
    dm01db08: CRS-4622: Oracle High Availability Services autostart is enabled.

    • Disable the Grid Infrastructure for autostart on the compute nodes if it is currently enabled for autostart.

    [root@dm01db01 ~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl disable crs
    dm01db01: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db02: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db03: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db04: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db05: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db06: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db07: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db08: CRS-4621: Oracle High Availability Services autostart is disabled.


    [root@dm01db01 ~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl config crs
    dm01db01: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db02: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db03: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db04: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db05: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db06: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db07: CRS-4621: Oracle High Availability Services autostart is disabled.
    dm01db08: CRS-4621: Oracle High Availability Services autostart is disabled.

    • Shutdown the Grid Infrastructure stack on the Exadata compute nodes

    This will take some time to come out
    [root@dm01db01 ~]#dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl stop crs


    [root@dm01db01 ~]# dcli -g dbs_group -l root /u01/app/11.2.0.4/grid/bin/crsctl check cluster
    dm01db01: CRS-4639: Could not contact Oracle High Availability Services
    dm01db01: CRS-4000: Command Check failed, or completed with errors.
    dm01db02: CRS-4639: Could not contact Oracle High Availability Services
    dm01db02: CRS-4000: Command Check failed, or completed with errors.
    dm01db03: CRS-4639: Could not contact Oracle High Availability Services
    dm01db03: CRS-4000: Command Check failed, or completed with errors.
    dm01db04: CRS-4639: Could not contact Oracle High Availability Services
    dm01db04: CRS-4000: Command Check failed, or completed with errors.
    dm01db05: CRS-4639: Could not contact Oracle High Availability Services
    dm01db05: CRS-4000: Command Check failed, or completed with errors.
    dm01db06: CRS-4639: Could not contact Oracle High Availability Services
    dm01db06: CRS-4000: Command Check failed, or completed with errors.
    dm01db07: CRS-4639: Could not contact Oracle High Availability Services
    dm01db07: CRS-4000: Command Check failed, or completed with errors.
    dm01db08: CRS-4639: Could not contact Oracle High Availability Services
    dm01db08: CRS-4000: Command Check failed, or completed with errors.

    • Ensure that the Grid Infrastructure stack has shutdown successfully on all the Compute nodes. 

    The following command returns no output if the Grid Infrastructure is shutdown:


    [root@dm01db01 ~]# dcli -g dbs_group -l root “ps -ef | grep diskmo[n]”

    • Disable email/SNMP alerts. The following command will show the current notification method:

    [root@dm01db01 ~]# dcli -g cell_group -l root “cellcli -e list cell attributes name,notificationMethod”
    dm01cel01: dm01cel01     none
    dm01cel02: dm01cel02     none
    dm01cel03: dm01cel03     none
    dm01cel04: dm01cel04     none
    dm01cel05: dm01cel05     none
    dm01cel06: dm01cel06     none
    dm01cel07: dm01cel07     none
    dm01cel08: dm01cel08     none
    dm01cel09: dm01cel09     none
    dm01cel10: dm01cel10     none
    dm01cel11: dm01cel11     none
    dm01cel12: dm01cel12     none
    dm01cel13: dm01cel13     none
    dm01cel14: dm01cel14     none

    • Now set the notification method to null to disable alerts:

    [root@dm01db01 ~]# dcli -g cell_group -l root “cellcli -e alter cell notificationMethod=none”
    dm01cel01: Cell dm01cel01 successfully altered
    dm01cel02: Cell dm01cel02 successfully altered
    dm01cel03: Cell dm01cel03 successfully altered
    dm01cel04: Cell dm01cel04 successfully altered
    dm01cel05: Cell dm01cel05 successfully altered
    dm01cel06: Cell dm01cel06 successfully altered
    dm01cel07: Cell dm01cel07 successfully altered
    dm01cel08: Cell dm01cel08 successfully altered
    dm01cel09: Cell dm01cel09 successfully altered
    dm01cel10: Cell dm01cel10 successfully altered
    dm01cel11: Cell dm01cel11 successfully altered
    dm01cel12: Cell dm01cel12 successfully altered
    dm01cel13: Cell dm01cel13 successfully altered
    dm01cel14: Cell dm01cel14 successfully altered

    • Shutdown cell services on all the cells:

    [root@dm01db01 ~]# dcli -g cell_group -l root “cellcli -e alter cell shutdown services all”
    dm01cel01:
    dm01cel01: Stopping the RS, CELLSRV, and MS services…
    dm01cel01: The SHUTDOWN of services was successful.
    dm01cel02:
    dm01cel02: Stopping the RS, CELLSRV, and MS services…
    dm01cel02: The SHUTDOWN of services was successful.
    dm01cel03:
    dm01cel03: Stopping the RS, CELLSRV, and MS services…
    dm01cel03: The SHUTDOWN of services was successful.
    dm01cel04:
    dm01cel04: Stopping the RS, CELLSRV, and MS services…
    dm01cel04: The SHUTDOWN of services was successful.
    dm01cel05:
    dm01cel05: Stopping the RS, CELLSRV, and MS services…
    dm01cel05: The SHUTDOWN of services was successful.
    dm01cel06:
    dm01cel06: Stopping the RS, CELLSRV, and MS services…
    dm01cel06: The SHUTDOWN of services was successful.
    dm01cel07:
    dm01cel07: Stopping the RS, CELLSRV, and MS services…
    dm01cel07: The SHUTDOWN of services was successful.
    dm01cel08:
    dm01cel08: Stopping the RS, CELLSRV, and MS services…
    dm01cel08: The SHUTDOWN of services was successful.
    dm01cel09:
    dm01cel09: Stopping the RS, CELLSRV, and MS services…
    dm01cel09: The SHUTDOWN of services was successful.
    dm01cel10:
    dm01cel10: Stopping the RS, CELLSRV, and MS services…
    dm01cel10: The SHUTDOWN of services was successful.
    dm01cel11:
    dm01cel11: Stopping the RS, CELLSRV, and MS services…
    dm01cel11: The SHUTDOWN of services was successful.
    dm01cel12:
    dm01cel12: Stopping the RS, CELLSRV, and MS services…
    dm01cel12: The SHUTDOWN of services was successful.
    dm01cel13:
    dm01cel13: Stopping the RS, CELLSRV, and MS services…
    dm01cel13: The SHUTDOWN of services was successful.
    dm01cel14:
    dm01cel14: Stopping the RS, CELLSRV, and MS services…
    dm01cel14: The SHUTDOWN of services was successful.
    [root@dm01db01 ~]#

    • Power off all storage cells:

    [root@dm01db01 ~]# dcli -g cell_group -l root poweroff
    or
    [root@dm01db01 ~]# dcli -g cell_group -l root ‘shutdown -h now’

    • Copy the /root/dbs_group to /root/dbs_group1:

    [root@dm01db01 ~]# cp /root/dbs_group /root/dbs_group1

    • Remove the “first compute node name” from the /root/dbs_group1 file (using any editor, e.g. vi).

    [root@dm01db01 ~]# vi /root/dbs_group1


    [root@dm01db01 ~]# cat /root/dbs_group1
    dm01db02
    dm01db03
    dm01db04
    dm01db05
    dm01db06
    dm01db07
    dm01db08

    • Power off all the compute nodes (except compute node # 1):

    [root@dm01db01 ~]# dcli -g dbs_group1 -l root poweroff
    or
    [root@dm01db01 ~]# dcli -g dbs_group1 -l root ‘shutdown -h now’

    • Power off the compute node #1:

    [root@dm01db01 ~]# poweroff


    Broadcast message from root@dm01db01.netsoftmate.com
            (/dev/pts/0) at 7:36 …


    The system is going down for power off NOW!

    • Exadata components are shutdown for maintenance.



    Conclusion
    In this article we have learnt how to Shutdown Exadata Database machine Rack for maintenance.