Backing up file systems on Oracle Exadata compute can be a daunting task if you are unaware of the prerequisites and best practices. To help you backup your file systems effectively and with least discrepancies, we are bringing you this interesting step-by-step guide on how to back up your file system using Oracle Exadata Sanpshot Based Backup of Compute Node to NFS Share

It is very important to take file system backup on Oracle Exadata compute nodes before we make any major changes to the operating system or critical software. On Oracle Exadata Compute nodes / (root) and /u01 file systems contains the operating system and GI/DB software respectively. These are the most critical file systems on Oracle Exadata computes.

By default, the / (root) and /u01 file system are sized 30GB and 100GB respectively.

Scenarios in which we must take file system backup are:

– Operating System Patching or Upgrade
– Grid Infrastructure Patching or Upgrade
– Database Patching or Upgrade
– Operating System configuration changes
– Increasing/decreasing file system size

In this article, we will demonstrate how to backup file system on Oracle Exadata Compute nodes running Linux Operating System to external storage NFS Share.

Environment Details

Exadata Model	X4-2 Half Rack HP 4 TB
Exadata Components	4 – Compute nodes, 7 – Storage cells & 2 – IB switches
Exadata Storage cells	DBM01CEL01 – DBM01CEL07
Exadata Compute nodes	DBM01DB01 – DBM01DB04
Exadata Software Version	19.2.3.0
Exadata DB Version	11.2.0.4.180717

Prerequisites

– Root user access on Compute nodes
– NFS mount with sufficient storage to storing file system backup

Current root and /u01 file system sizes

[root@ip01db01 ~]# df -h / /u01

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/VGExaDb-LVDbSys1 59G 39G 18G 70% /

/dev/mapper/VGExaDb-LVDbOra1 197G 171G 17G 92% /u01

NFS share details

10.10.10.1:/nfs/backup/

1. As root user, log in to the Exadata Compute node you wish to backup

[root@ip01db01 ~]# id root

uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)

2. Create a mount point to mount NFS share

[root@dm01db01 ~]# mkdir -p /mnt/backup

[root@dm01db01 ~]# mount -t nfs -o rw,intr,soft,proto=tcp,nolock 10.10.10.1:/nfs/backup/ /mnt/backup

3. Determine the file system type extension root and u01 file system

[root@ip01db01 ~]# mount -l

sysfs on /sys type sysfs (rw,relatime)

proc on /proc type proc (rw,relatime)

devtmpfs on /dev type devtmpfs (rw,nosuid,size=131804372k,nr_inodes=32951093,mode=755)

securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)

tmpfs on /dev/shm type tmpfs (rw,size=264225792k)

/dev/mapper/VGExaDb-LVDbSys1 on / type ext3 (rw,relatime,errors=continue,barrier=1,data=ordered) [DBSYS]

/dev/mapper/VGExaDb-LVDbOra1 on /u01 type ext3 (rw,nodev,relatime,errors=continue,barrier=1,data=ordered) [DBORA]

/dev/sda1 on /boot type ext3 (rw,nodev,relatime,errors=continue,barrier=1,data=ordered) [BOOT]

4. Note down the file system type for root and /u01: ext3. Take a snapshot-based of the root and u01 directories, Label the snapshot and Mount the snapshot

[root@dm01db01 ~]# lvcreate -L5G -s -n root_snap /dev/VGExaDb/LVDbSys1

Logical volume “root_snap” created.

[root@dm01db01 ~]# lvcreate -L5G -s -n u01_snap /dev/VGExaDb/LVDbOra1

Logical volume “u01_snap” created.

[root@dm01db01 ~]# e2label /dev/VGExaDb/root_snap DBSYS_SNAP

[root@dm01db01 ~]# e2label /dev/VGExaDb/u01_snap DBORA_SNAP

[root@dm01db01 ~]# mkdir -p /mnt/snap/root

[root@dm01db01 ~]# mkdir -p /mnt/snap/u01

[root@dm01db01 ~]# mount /dev/VGExaDb/root_snap /mnt/snap/root -t ext3

[root@dm01db01 ~]# mount /dev/VGExaDb/u01_snap /mnt/snap/u01 -t ext3

[root@dm01db01 ~]# df -h /mnt/snap/root

[root@dm01db01 ~]# df -h /mnt/snap/u01

5. Change to the directory for the backup and Create the backup file

[root@dm01db01 ~]# cd /mnt/backup

[root@dm01db01 ~]# tar -pjcvf /mnt/backup/mybackup.tar.bz2 * /boot –exclude /mnt/backup/mybackup.tar.bz2 > /tmp/backup_tar.stdout 2> /tmp/backup_tar.stderr

6. Monitor the /tmp/backup_tar.stderr file for errors. Errors such as failing to tar open sockets can be ignored.

7. Unmount the snapshots and remove the snapshots for the root and /01 directories.

[root@dm01db01 ~]# cd /

[root@dm01db01 /]# umount /mnt/snap/u01

[root@dm01db01 /]# umount /mnt/snap/root

[root@dm01db01 /]# df -h /mnt/snap/u01

[root@dm01db01 /]# df -h /mnt/snap/root

[root@dm01db01 /]# ls -l /mnt/snap

[root@dm01db01 /]# rm -rf /mnt/snap

[root@dm01db01 /]# lvremove /dev/VGExaDb/u01_snap

Do you really want to remove active logical volume u01_snap? [y/n]: y

Logical volume “u01_snap” successfully removed

[root@dm01db01 /]# lvremove /dev/VGExaDb/root_snap

Do you really want to remove active logical volume root_snap? [y/n]: y

Logical volume “root_snap” successfully removed

8. Unmount the NFS share

[root@dm01db01 /] umount /mnt/backup

Repeat the above steps on the remaining compute nodes to backup root & u01 file system

We hope this article helps you smoothly backup your file system on Oracle Exadata Compute node running Oracle Linux using NFS Share. Stay tuned for more step-by-step guides on implementing and using Oracle Database Systems exclusively on Netsoftmate

Netsoftmate provides the best standard services when it comes to Oracle databases management, covering all comples database products. Sign-up for a free 30 minutes call by clicking on the link below –

Click here and fill the contact us form for a free 30 minutes consultation

Uncategorized

Change root User Password on Exadata Infiniband Switch

By Netsoftmate on

February 18, 2019

- Uncategorized

I was working on changing password for the administrative user accounts on all Exadata Components. I encountered a strange issue while changing the root password on Infiniband Switch. We were unable to change the root password on IB Siwtch using command line method. We used couple different command line methods to change the root password on IB switches but all of them failed. This could be a BUG, firmware issue or something else.

In this article we demonstrate how to change the root password on an Exadata infiniband switch using Browser User Interface.

Issue 1: Using passwd command

Tried to change the root user password using passwd command using dcli. This method assumes you are have ssh equivalence setup from compute node 1. As you can see the command failed saying to use the ILOM shell. In the past I have used the same command successfully to change the root password on IB Switches.

[root@dm01db01 ~]# dcli -g ibswitch_group -l root “echo welcome1 | passwd –stdin root”
dm01sw-ibb01: This command should not be used for ILOM users.
dm01sw-ibb01: Please use ILOM shell to handle password for this user.
dm01sw-ibb01: Example:
dm01sw-ibb01: -> set /SP/users/root password
dm01sw-ibb01:
dm01sw-iba01: This command should not be used for ILOM users.
dm01sw-iba01: Please use ILOM shell to handle password for this user.
dm01sw-iba01: Example:
dm01sw-iba01: -> set /SP/users/root password
dm01sw-iba01:

So I decided to login to the IB switch directly and use the passwd command instead of running from dcli. The passwd command fail again with the same error.

[root@dm01sw-iba01 ~]# ssh dm01sw-ibb01
You are now logged in to the root shell.
It is recommended to use ILOM shell instead of root shell.
All usage should be restricted to documented commands and documented
config files.
To view the list of documented commands, use “help” at linux prompt.

[root@dm01sw-ibb01 ~]# hostname
dm01sw-ibb01

[root@dm01sw-iba01 ~]# passwd root
This command should not be used for ILOM users.
Please use ILOM shell to handle password for this user.
Example:
-> set /SP/users/root password

eBook - Oracle Exadata X8M Patching Recipes | Netsoftmate

Issue 2: Using ILOM Shell

As the passwd command failed asking to use the ILOM shell, I login to the IB switch as ilom-admin and executed the change password command. What I see is, the password change command failed at ILOM prompt as well.

[root@dm01sw-iba01 ~]# su – ilom-admin

Oracle(R) Integrated Lights Out Manager
Version 2.2.7-1 ILOM 3.2.6 r118629
Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
Warning: HTTPS certificate is set to factory default.

Hostname: dm01sw-iba01

-> set /SP/users/root welcome1
set: Invalid command syntax
Usage: set [-script] [target] <property>=<value> [<property>=<value>…]

Solution: Using Browser User Interface

I have decided to use the BUI to change the password.

Steps:

Open a Browser and enter the IB Switch hostname or IP address

https://dm01sw-ibb01.netsoftmate.com

Accept the security warning and proceed to connect to the IB Switch

Enter the username and password to connect to the IB Switch

This show the summary page

On the left Pan, expand ILOM administration and select User Management

Click on User Accounts, Select root user and click on edit button

Enter the new password and confirm and Finally click on the Save button to change the password.

To Verify the new password, open a Putty session and ssh to IB Switch using new password.

[root@dm01db01 ~]# ssh dm01sw-ibb01
Password:
You are now logged in to the root shell.
It is recommended to use ILOM shell instead of root shell.
All usage should be restricted to documented commands and documented
config files.
To view the list of documented commands, use “help” at linux prompt.

[root@dm01sw-ibb01 ~]# hostname
dm01sw-ibb01

Conclusion

In this article we have learned how to change the root password on Infiniband Switch using Browser User Interface when the command line option doesn’t work.

Oracle Exadata Database Machine Health Check – Exachk 18c

By Netsoftmate on

May 20, 2018

- Uncategorized

Oracle has released Exachk utility 18c on May 18th, 2018. Let’s quickly check if there are differences in Exachk 18c or it is similar to Exachk 12c.

Download latest Exachk 18c utility from MOS note:
Oracle Exadata Database Machine exachk or HealthCheck (Doc ID 1070954.1)

Changes in Exachk 18.2 can be found at:
https://docs.oracle.com/cd/E96145_01/OEXUG/changes-in-this-release-18-2-0.htm#OEXUG-GUID-88FCFBC6-C647-47D3-898C-F4C712117B8B

Steps to Execute Exachk 18c on Exadata Database Machine

Download the latest Exachk from MOS note. Here I am downloading Exachk 18c.

Download Completed

Using WinSCP copy the exachk.zip file to Exadata Compute node

Copy completed. List the Exachk file on Compute node

Unzip the Exachk zip file

Verify Exachk version

Execute Exachk Health by running the following command

Exachk execution completed

Review the Exachk report and take necessary action

Conclusion
In this article we have learned how to execute Oracle Exadata Database Machine health Check using Exachk 18c. Using Exachk 18c is NO different than it’s previous releases.

Exadata Setup SSH Equivalence

By Netsoftmate on

May 20, 2018

- Uncategorized

You want to execute Operating System or Exadata commands on multiple Exadata Compute nodes and Storage Cell in parallel. To accomplish this you must setup passwordless SSH across compute nodes and storage cells.

If SSH equivalence is NOT setup and you execute the dcli command you will see the follow messages. This mean the SSH equivalence is not configured.

[root@dm01db01 ~]# dcli -g dbs_group -l root ‘uptime’
The authenticity of host ‘dm01db03 (10.10.10.195)’ can’t be established.
RSA key fingerprint is 40:81:3c:6d:ef:e7:1f:d7:a0:df:eb:f5:ea:92:a5:db.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db05 (10.10.10.197)’ can’t be established.
RSA key fingerprint is 1b:95:47:0b:92:b4:13:9f:55:b7:a3:2a:56:27:9f:1c.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db02 (10.10.10.194)’ can’t be established.
RSA key fingerprint is e1:0d:90:46:16:88:74:01:02:5a:11:90:63:b1:6b:1c.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db01 (10.10.10.193)’ can’t be established.
RSA key fingerprint is 2b:6f:43:4b:86:29:bb:ed:a6:03:c5:34:75:cf:45:34.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db04 (10.10.10.196)’ can’t be established.
RSA key fingerprint is 44:a7:ad:65:c3:1c:fb:0b:0b:28:2c:b6:a5:f3:59:99.
Are you sure you want to continue connecting (yes/no)? The authenticity of host ‘dm01db07 (10.10.10.199)’ can’t be established.
RSA key fingerprint is 25:5f:9a:e6:a4:7a:13:ba:e2:e7:7d:2e:79:53:49:2b.
Are you sure you want to continue connecting (yes/no)? root@dm01db06’s password: root@dm01db08’s password:

In this article we will demonstrate how to setup SSH equivalence on Exadata Database Machine.

Steps to Setup SSH Equivalence

1. Create the following files if doesn’t exist

[root@dm01db08 ~]# cat dbs_group
dm01db01
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08

[root@dm01db08 ~]# cat cell_group
dm01cel01
dm01cel02
dm01cel03
dm01cel04
dm01cel05
dm01cel06
dm01cel07

[root@dm01db08 ~]# cat all_group
dm01db01
dm01db02
dm01db03
dm01db04
dm01db05
dm01db06
dm01db07
dm01db08
dm01cel01
dm01cel02
dm01cel03
dm01cel04
dm01cel05
dm01cel06
dm01cel07
dm01sw-iba01
dm01sw-ibb01

2. Navigate to Support directory on Compute node 1 as shown below

[root@dm01db01 ~]# cd /opt/oracle.SupportTools/

3. Oracle has provided a script *setup_ssh_eq.sh* to configure SSH equivalence across Exadata components. Execute the script as shown below. Here we are setting the SSH equivalence for root user

[root@dm01db01 oracle.SupportTools]# ./setup_ssh_eq.sh ~/all_group root welcome1

/root/.ssh/id_dsa already exists.

Overwrite (y/n)?

/root/.ssh/id_rsa already exists.

Overwrite (y/n)?

spawn dcli -c dm01db01 -l root -k

dm01db01: ssh key already exists