Good Contents Are Everywhere, But Here, We Deliver The Best of The Best.Please Hold on!
Cloud Services, Database Management Services, Oracle Exadata, Training

 

Backing up file systems on Oracle Exadata compute can be a daunting task if you are unaware of the prerequisites and best practices. To help you backup your file systems effectively and with least discrepancies, we are bringing you this interesting step-by-step guide on how to back up your file system using Oracle Exadata Sanpshot Based Backup of Compute Node to NFS Share    

 

It is very important to take file system backup on Oracle Exadata compute nodes before we make any major changes to the operating system or critical software. On Oracle Exadata Compute nodes / (root) and /u01 file systems contains the operating system and GI/DB software respectively. These are the most critical file systems on Oracle Exadata computes.

 

By default, the / (root) and /u01 file system are sized 30GB and 100GB respectively.

 

Scenarios in which we must take file system backup are:

 

  • – Operating System Patching or Upgrade
  •  
  • – Grid Infrastructure Patching or Upgrade
  •  
  • – Database Patching or Upgrade
  •  
  • – Operating System configuration changes
  •  
  • – Increasing/decreasing file system size

 

In this article, we will demonstrate how to backup file system on Oracle Exadata Compute nodes running Linux Operating System to external storage NFS Share.

eBook - Oracle Exadata X8M Patching Recipes | Netsoftmate

Environment Details

 

Exadata Model

X4-2 Half Rack HP 4 TB

Exadata Components

4 – Compute nodes, 7 – Storage cells & 2 – IB switches

Exadata Storage cells

DBM01CEL01 – DBM01CEL07

Exadata Compute nodes

DBM01DB01 – DBM01DB04

Exadata Software Version

19.2.3.0

Exadata DB Version

11.2.0.4.180717

 

 

Prerequisites

 

  • – Root user access on Compute nodes
  •  
  • – NFS mount with sufficient storage to storing file system backup




Current root and /u01 file system sizes

 

[root@ip01db01 ~]# df -h / /u01

Filesystem Size  Used Avail Use% Mounted on

/dev/mapper/VGExaDb-LVDbSys1   59G   39G   18G  70% /

/dev/mapper/VGExaDb-LVDbOra1  197G  171G   17G  92% /u01



NFS share details

 

10.10.10.1:/nfs/backup/

 

  • 1. As root user, log in to the Exadata Compute node you wish to backup

 

[root@ip01db01 ~]# id root

uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)


 

  • 2. Create a mount point to mount NFS share

 

[root@dm01db01 ~]# mkdir -p /mnt/backup

[root@dm01db01 ~]# mount -t nfs -o rw,intr,soft,proto=tcp,nolock 10.10.10.1:/nfs/backup/ /mnt/backup

 


3. Determine the file system type extension root and u01 file system

 

[root@ip01db01 ~]# mount -l

sysfs on /sys type sysfs (rw,relatime)

proc on /proc type proc (rw,relatime)

devtmpfs on /dev type devtmpfs (rw,nosuid,size=131804372k,nr_inodes=32951093,mode=755)

securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)

tmpfs on /dev/shm type tmpfs (rw,size=264225792k)

/dev/mapper/VGExaDb-LVDbSys1 on / type ext3 (rw,relatime,errors=continue,barrier=1,data=ordered) [DBSYS]

/dev/mapper/VGExaDb-LVDbOra1 on /u01 type ext3 (rw,nodev,relatime,errors=continue,barrier=1,data=ordered) [DBORA]

/dev/sda1 on /boot type ext3 (rw,nodev,relatime,errors=continue,barrier=1,data=ordered) [BOOT]

 

 

4.  Note down the file system type for root and /u01: ext3. Take a snapshot-based of the root and u01 directories, Label the snapshot and Mount the snapshot

 

[root@dm01db01 ~]# lvcreate -L5G -s -n root_snap /dev/VGExaDb/LVDbSys1

  Logical volume “root_snap” created.

 

[root@dm01db01 ~]# lvcreate -L5G -s -n u01_snap /dev/VGExaDb/LVDbOra1

  Logical volume “u01_snap” created.



[root@dm01db01 ~]# e2label /dev/VGExaDb/root_snap DBSYS_SNAP

[root@dm01db01 ~]# e2label /dev/VGExaDb/u01_snap DBORA_SNAP

 

[root@dm01db01 ~]# mkdir -p /mnt/snap/root

[root@dm01db01 ~]# mkdir -p /mnt/snap/u01

 

 

[root@dm01db01 ~]# mount /dev/VGExaDb/root_snap /mnt/snap/root -t ext3

[root@dm01db01 ~]# mount /dev/VGExaDb/u01_snap /mnt/snap/u01 -t ext3

 

 

[root@dm01db01 ~]# df -h /mnt/snap/root

[root@dm01db01 ~]# df -h /mnt/snap/u01

 

 

  • 5. Change to the directory for the backup and Create the backup file

 

[root@dm01db01 ~]#  cd /mnt/backup

 

[root@dm01db01 ~]#  tar -pjcvf /mnt/backup/mybackup.tar.bz2 * /boot –exclude /mnt/backup/mybackup.tar.bz2 > /tmp/backup_tar.stdout 2> /tmp/backup_tar.stderr

 

  • 6. Monitor the /tmp/backup_tar.stderr file for errors. Errors such as failing to tar open sockets can be ignored.

 

  • 7. Unmount the snapshots and remove the snapshots for the root and /01 directories.

 

[root@dm01db01 ~]# cd /

 

[root@dm01db01 /]# umount /mnt/snap/u01

 

[root@dm01db01 /]# umount /mnt/snap/root

 

[root@dm01db01 /]# df -h /mnt/snap/u01

[root@dm01db01 /]# df -h /mnt/snap/root

 

[root@dm01db01 /]# ls -l /mnt/snap

 

[root@dm01db01 /]# rm -rf /mnt/snap

 

 

[root@dm01db01 /]# lvremove /dev/VGExaDb/u01_snap

Do you really want to remove active logical volume u01_snap? [y/n]: y

  Logical volume “u01_snap” successfully removed


 

[root@dm01db01 /]# lvremove /dev/VGExaDb/root_snap

Do you really want to remove active logical volume root_snap? [y/n]: y

  Logical volume “root_snap” successfully removed


 

  • 8. Unmount the NFS share

 

[root@dm01db01 /] umount /mnt/backup


 

  • Repeat the above steps on the remaining compute nodes to backup root & u01 file system



We hope this article helps you smoothly backup your file system on Oracle Exadata Compute node running Oracle Linux using NFS Share. Stay tuned for more step-by-step guides on implementing and using Oracle Database Systems exclusively on Netsoftmate




Netsoftmate provides the best standard services when it comes to Oracle databases management, covering all comples database products. Sign-up for a free 30 minutes call by clicking on the link below –



Click here and fill the contact us form for a free 30 minutes consultation

 

3

Uncategorized
I was working on changing password for the administrative user accounts on all Exadata Components. I encountered a strange issue while changing the root password on Infiniband Switch. We were unable to change the root password on IB Siwtch using command line method. We used couple different command line methods to change the root password on IB switches but all of them failed. This could be a BUG, firmware issue or something else.

In this article we demonstrate how to change the root password on an Exadata infiniband switch using Browser User Interface.

Issue 1: Using passwd command

Tried to change the root user password using passwd command using dcli. This method assumes you are have ssh equivalence setup from compute node 1. As you can see the command failed saying to use the ILOM shell. In the past I have used the same command successfully to change the root password on IB Switches.

[root@dm01db01 ~]#  dcli -g ibswitch_group -l root “echo welcome1 | passwd –stdin root”
dm01sw-ibb01: This command should not be used for ILOM users.
dm01sw-ibb01: Please use ILOM shell to handle password for this user.
dm01sw-ibb01: Example:
dm01sw-ibb01: -> set /SP/users/root password
dm01sw-ibb01:
dm01sw-iba01: This command should not be used for ILOM users.
dm01sw-iba01: Please use ILOM shell to handle password for this user.
dm01sw-iba01: Example:
dm01sw-iba01: -> set /SP/users/root password
dm01sw-iba01:


So I decided to login to the IB switch directly and use the passwd command instead of running from dcli. The passwd command fail again with the same error.

[root@dm01sw-iba01 ~]# ssh dm01sw-ibb01
You are now logged in to the root shell.
It is recommended to use ILOM shell instead of root shell.
All usage should be restricted to documented commands and documented
config files.
To view the list of documented commands, use “help” at linux prompt.

[root@dm01sw-ibb01 ~]# hostname
dm01sw-ibb01

[root@dm01sw-iba01 ~]# passwd root
This command should not be used for ILOM users.
Please use ILOM shell to handle password for this user.
Example:
   -> set /SP/users/root password



eBook - Oracle Exadata X8M Patching Recipes | Netsoftmate

Issue 2: Using ILOM Shell

As the passwd command failed asking to use the ILOM shell, I login to the IB switch as ilom-admin and executed the change password command. What I see is, the password change command failed at ILOM prompt as well.

[root@dm01sw-iba01 ~]# su – ilom-admin

Oracle(R) Integrated Lights Out Manager
Version 2.2.7-1 ILOM 3.2.6 r118629
Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
Warning: HTTPS certificate is set to factory default.
 

Hostname: dm01sw-iba01

-> set /SP/users/root welcome1
set: Invalid command syntax
Usage: set [-script] [target] <property>=<value> [<property>=<value>…]



 

Solution: Using Browser User Interface

I have decided to use the BUI to change the password.

Steps:

  • Open a Browser and enter the IB Switch hostname or IP address
https://dm01sw-ibb01.netsoftmate.com
  • Accept the security warning and proceed to connect to the IB Switch
  • Enter the username and password to connect to the IB Switch

  • This show the summary page

  • On the left Pan, expand ILOM administration and select User Management

  • Click on  User Accounts, Select root user and click on edit button

  • Enter the new password and confirm and Finally click on the Save button to change the password.

  • To Verify the new password, open a Putty session and ssh to IB Switch using new password.
[root@dm01db01 ~]# ssh dm01sw-ibb01
Password:
You are now logged in to the root shell.
It is recommended to use ILOM shell instead of root shell.
All usage should be restricted to documented commands and documented
config files.
To view the list of documented commands, use “help” at linux prompt.

[root@dm01sw-ibb01 ~]# hostname
dm01sw-ibb01



Conclusion

In this article we have learned how to change the root password on Infiniband Switch using Browser User Interface when the command line option doesn’t work.
0

Oracle has released Exachk utility 18c on May 18th, 2018. Let’s quickly check if there are differences in Exachk 18c or it is similar to Exachk 12c.

Download latest Exachk 18c utility from MOS note:
Oracle Exadata Database Machine exachk or HealthCheck (Doc ID 1070954.1)

Changes in Exachk 18.2 can be found at:
https://docs.oracle.com/cd/E96145_01/OEXUG/changes-in-this-release-18-2-0.htm#OEXUG-GUID-88FCFBC6-C647-47D3-898C-F4C712117B8B

Steps to Execute Exachk 18c on Exadata Database Machine


Download the latest Exachk from MOS note. Here I am downloading Exachk 18c.

Download Completed

Using WinSCP copy the exachk.zip file to Exadata Compute node



Copy completed. List the Exachk file on Compute node

Unzip the Exachk zip file

Verify Exachk version

Execute Exachk Health by running the following command

Exachk execution completed

Review the Exachk report and take necessary action



Conclusion
In this article we have learned how to execute Oracle Exadata Database Machine health Check using Exachk 18c. Using Exachk 18c is NO different than it’s previous releases.

0

You want to execute Operating System or Exadata commands on multiple Exadata Compute nodes and Storage Cell in parallel. To accomplish this you must setup passwordless SSH across compute nodes and storage cells.

If SSH equivalence is NOT setup and you execute the dcli command you will see the follow messages. This mean the SSH equivalence is not configured.

[root@dm01db01 ~]# dcli -g dbs_group -l root ‘uptime’
The authenticity of host ‘dm01db03 (10.10.10.195)’ can’t be established.
RSA key fingerprint is 40:81:3c:6d:ef:e7:1f:d7:a0:df:eb:f5:ea:92:a5:db.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db05 (10.10.10.197)’ can’t be established.
RSA key fingerprint is 1b:95:47:0b:92:b4:13:9f:55:b7:a3:2a:56:27:9f:1c.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db02 (10.10.10.194)’ can’t be established.
RSA key fingerprint is e1:0d:90:46:16:88:74:01:02:5a:11:90:63:b1:6b:1c.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db01 (10.10.10.193)’ can’t be established.
RSA key fingerprint is 2b:6f:43:4b:86:29:bb:ed:a6:03:c5:34:75:cf:45:34.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db04 (10.10.10.196)’ can’t be established.
RSA key fingerprint is 44:a7:ad:65:c3:1c:fb:0b:0b:28:2c:b6:a5:f3:59:99.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db07 (10.10.10.199)’ can’t be established.
RSA key fingerprint is 25:5f:9a:e6:a4:7a:13:ba:e2:e7:7d:2e:79:53:49:2b.
Are you sure you want to continue connecting (yes/no)? root@dm01db06’s password: root@dm01db08’s password:

In this article we will demonstrate how to setup SSH equivalence on Exadata Database Machine.


Steps to Setup SSH Equivalence

1. Create the following files if doesn’t exist

[root@dm01db08 ~]# cat dbs_group
dm01db01
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08

[root@dm01db08 ~]# cat cell_group
dm01cel01
dm01cel02
dm01cel03
dm01cel04
dm01cel05
dm01cel06
dm01cel07

[root@dm01db08 ~]# cat all_group
dm01db01
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08
dm01cel01
dm01cel02
dm01cel03
dm01cel04
dm01cel05
dm01cel06
dm01cel07
dm01sw-iba01
dm01sw-ibb01

2. Navigate to Support directory on Compute node 1 as shown below

[root@dm01db01 ~]# cd /opt/oracle.SupportTools/

3. Oracle has provided a script *setup_ssh_eq.sh* to configure SSH equivalence across Exadata components. Execute the script as shown below. Here we are setting the SSH equivalence for root user

[root@dm01db01 oracle.SupportTools]# ./setup_ssh_eq.sh ~/all_group root welcome1
/root/.ssh/id_dsa already exists.
Overwrite (y/n)?
/root/.ssh/id_rsa already exists.
Overwrite (y/n)?
spawn dcli -c dm01db01 -l root -k
dm01db01: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db02 -l root -k
dm01db02: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db03 -l root -k
dm01db03: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db04 -l root -k
dm01db04: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db05 -l root -k
dm01db05: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db06 -l root -k
dm01db06: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db07 -l root -k
dm01db07: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01db08 -l root -k
dm01db08: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel01 -l root -k
dm01cel01: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel02 -l root -k
dm01cel02: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel03 -l root -k
dm01cel03: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel04 -l root -k
dm01cel04: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel05 -l root -k
dm01cel05: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel06 -l root -k
dm01cel06: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01cel07 -l root -k
dm01cel07: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01sw-iba01 -l root -k
dm01sw-iba01: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””
spawn dcli -c dm01sw-ibb01 -l root -k
dm01sw-ibb01: ssh key already exists
expect: spawn id exp4 not open
    while executing
“expect “*?assword:*””

4. Verify SSH equivalence is working fine

[root@dm01db08 ~]# dcli -g ~/all_group -l root ‘uptime’
dm01db01: 09:16:41 up 21 days, 15:47,  1 user,  load average: 1.80, 3.02, 3.35
dm01db02: 09:16:41 up 21 days, 15:38,  0 users,  load average: 2.93, 2.44, 2.37
dm01db03: 09:16:41 up 21 days, 15:19,  0 users,  load average: 2.16, 2.27, 2.77
dm01db04: 09:16:41 up 21 days, 15:12,  0 users,  load average: 4.07, 4.33, 4.14
dm01db05: 09:16:41 up 21 days, 15:09,  0 users,  load average: 2.45, 2.82, 2.75
dm01db06: 09:16:41 up 21 days, 15:06,  0 users,  load average: 1.70, 2.04, 2.60
dm01db07: 09:16:41 up 21 days, 15:02,  0 users,  load average: 6.39, 4.46, 4.20
dm01db08: 09:16:41 up 21 days, 14:59,  1 user,  load average: 1.66, 1.81, 1.97
dm01cel01: 09:16:41 up 203 days, 19:00,  0 users,  load average: 1.40, 1.97, 2.21
dm01cel02: 09:16:41 up 203 days, 18:59,  0 users,  load average: 1.52, 2.08, 2.38
dm01cel03: 09:16:41 up 203 days, 18:59,  0 users,  load average: 1.00, 1.71, 2.02
dm01cel04: 09:16:41 up 203 days, 18:59,  0 users,  load average: 1.08, 1.59, 1.92
dm01cel05: 09:16:41 up 203 days, 18:59,  0 users,  load average: 1.24, 1.53, 1.82
dm01cel06: 09:16:41 up 203 days, 18:59,  0 users,  load average: 1.09, 1.60, 1.96
dm01cel07: 09:16:41 up 203 days, 19:00,  0 users,  load average: 1.01, 1.37, 1.60
dm01sw-iba01: 09:16:42 up 539 days,  6:21,  0 users,  load average: 0.79, 0.99, 1.07
dm01sw-ibb01: 14:49:54 up 539 days,  9:43,  0 users,  load average: 1.26, 1.44, 1.41

[root@dm01db08 ~]# dcli -g dbs_group -l root ‘imageinfo | grep “Image version”‘
dm01db01: Image version: 12.1.2.3.6.170713
dm01db02: Image version: 12.1.2.3.6.170713
dm01db03: Image version: 12.1.2.3.6.170713
dm01db04: Image version: 12.1.2.3.6.170713
dm01db05: Image version: 12.1.2.3.6.170713
dm01db06: Image version: 12.1.2.3.6.170713
dm01db07: Image version: 12.1.2.3.6.170713
dm01db08: Image version: 12.1.2.3.6.170713



[root@dm01db08 ~]# dcli -g cell_group -l root ‘imageinfo | grep “Active image version”‘
dm01cel01: Active image version: 12.1.2.3.6.170713
dm01cel02: Active image version: 12.1.2.3.6.170713
dm01cel03: Active image version: 12.1.2.3.6.170713
dm01cel04: Active image version: 12.1.2.3.6.170713
dm01cel05: Active image version: 12.1.2.3.6.170713
dm01cel06: Active image version: 12.1.2.3.6.170713
dm01cel07: Active image version: 12.1.2.3.6.170713

[root@dm01db08 ~]# ssh dm01sw-iba01 version
SUN DCS 36p version: 2.1.8-1
Build time: Sep 18 2015 10:26:47
SP board info:
Manufacturing Date: 2015.05.13
Serial Number: “NCDKO0980”
Hardware Revision: 0x0200
Firmware Revision: 0x0000
BIOS version: SUN0R100
BIOS date: 06/22/2010

Conclusion

In this article we have learned how to configure SSH equivalence on Exadata Database Machine. Using the setup_ssh_eq.sh script is very easy setup SSH equivalence.

0


Oracle provides “Exachk” utility to conduct a comprehensive Exadata Health Check on Exadata Database Machine to validate hardware, firmware and configuration.


Exachk Utility is available for Oracle engineered systems such as Exadata (V2 and above), Exalogic, Exalytics, SuperCluster, MiniCluster, ZDLRA & Big Data. Exachk utility performs the following checks:
  • Configuration checks for Compute nodes, Storage cells and InfiniBand Switches
  • Grid Infrastructure, Database and ASM and Operating System software checks
  • MAA Scorecard which conducts an automatic MAA Review
  • Exadata Software Planner, Software prechecks, Exadata and Database Critical Issue alerts


It is recommended to execute the latest exachk at the following situation:
  • Monthly
  • Before any planned maintenance activity
  • Immediately after completion of planned maintenance activity
  • Immediately after an outage or incident


Steps to Exadata Health Exachk Using Exachk Utility

  • Download latest Exachk utility from the MOS note. As of writing the latest Exachk verion available is “12.2.0.1.4_20171212”
Oracle Exadata Database Machine exachk or Health Check (Doc ID 1070954.1)


Note: It is recommended to use latest Exachk to perform Exadata Health Check



  • As root user, create ‘Exachk’ directory on compute node 1 as follows
[root@dm01db01 ~]# cd /root
[root@dm01db01 ~]# mkdir Exachk

  • Using Winscp Copy the Downloaded Exachk utility from your desktop/laptop to the Exadata compute node 1 under /root/Exachk





  • As root user, Login to Exadata Compute node 1 and unzip the Exachk utility
[root@dm01db01 ~]# cd /root/Exachk/


[root@dm01db01 Exachk]# ls -ltr
total 112576
-rw-r–r– 1 root root 115158363 Apr 10 05:11 exachk.zip


[root@dm01db01 Exachk]# unzip exachk.zip





  • Ensure that the SSH is setup across all Compute nodes, Storage cells and Ibswitches
[root@dm01db01 Exachk]# dcli -g ~/all_group -l root ‘uptime’




To Setup SSH across the cluster, use the following command:


[root@dm01db01 ~]# cd /opt/oracle.SupportTools/
[root@dm01db01 oracle.SupportTools]# ./setup_ssh_eq.sh ~/all_group root welcome1

  • As root user, Execute the Exachk utility
[root@dm01db01 ~]# cd /root/Exachk/


[root@dm01db01 Exachk]# ls -ltr


[root@dm01db01 Exachk]# ./exachk





Depending on the Exadata Cluster Size and number of databases it may take several minutes to complete Exachk execution.
  • Using Winscp, copy the Exachk zip file and/or html file to your desktop/laptop to review
  • Open the html file, review it and take necessary action if necessary
  • Under the table of contents there are different component listed. Look out for the CRITICAL and FAIL components.
Click on the ‘view’ hyperlink for more details and the recommendation to fix the problem.




MAA Scorecard




Conclusion

In this article we have learned to perform complete Exadata Stack Health Check using Exachk utility. Exachk Utility is available for Oracle engineered systems such as Exadata (V2 and above), Exalogi, Exalytics, SuperCluster, MiniCluster, ZDLRA & Big Data.
4