ExaCC: Update of Exadata System Software using patchmgr

In this blog post, I will demonstrate how to use the command-line utility DB Server Patch Utility (patchmgr) to update the Exadata software of DomU environments to a newer version.

Exadata System Software Updates

General

On the DomU level, an Exadata System Software Update includes the following updates for a database VM server (also called Guest VM).

  • Oracle Linux operating system
  • Exadata system software

The available and supported Exadata System Software Updates for an ExaCC Gen2 environment are listed in My Oracle Support note Exadata Cloud Service Software Versions (Doc ID 2333222.1).

Updates of the physical storage and database servers are performed by Oracle Cloud Operations.

Download

Exadata System Software Updates are uploaded to the Control Plane Servers by Oracle Cloud Operations. To see and download these updates, you can use the script exadata_updates.sh, which is available in My Oracle Support note How to Obtain the Compute Node Exadata System Software Update for Exadata Cloud@Customer (Doc ID 2730739.1).

After the download, upload the script file to one of the VM servers.

$> chmod +x exadata_updates.sh
$> ./exadata_updates.sh -list
2021-08-14 09:11:34 +0200        :INFO   : Available Exadata Compute update(s):
20.1.12.0.0.210717
20.1.11.0.0.210608
20.1.10.0.0.210506
20.1.8.0.0.210317
20.1.9.0.0.210416
20.1.6.0.0.210113
20.1.7.0.0.210205

In the next step, the required update (e.g. 20.1.12.0.0.210717) can be downloaded to the local server.

$> ./exadata_updates.sh -get 20.1.12.0.0.210717
2021-08-14 09:13:00 +0200        :INFO   : Downloading Exadata Compute update: 20.1.12.0.0.210717
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1679M  100 1679M    0     0   109M      0  0:00:15  0:00:15 --:--:--  113M
2021-08-14 09:13:16 +0200        :SUCCESS: Download complete: /tmp/exadata_compute_20.1.12.0.0.210717.zip

Choose Launch Server(s)

To run patchmgr, a Linux-based system is required. You can either run patchmgr on the VM cluster itself or on an independent system. If you use the latter approach, you can run the update for all VM cluster nodes in one step.

If you choose to run patchmgr on the VM cluster itself, the node where patchmgr is running cannot be part of the patching. After the first run is complete, start patchmgr from a different node to patch the remaining node.

Upload the Exadata System Software Update to the launch server and unzip it.

$> unzip exadata_compute_20.1.12.0.0.210717.zip
Archive:  exadata_compute_20.1.12.0.0.210717.zip
   creating: 20.1.12.0.0.210717/
 extracting: 20.1.12.0.0.210717/dbserver.patch.zip
 extracting: 20.1.12.0.0.210717/exadata_ol7_20.1.12.0.0.210717_Linux-x86-64.zip

Update patchmgr

Although an Exadata System Software Update contains a version of patchmgr (dbserver.patch.zip), it is recommended to use the latest version of patchmgr. Download patch 21634633 from My Oracle Support.

After the download, upload the patch to the launch server and unzip it there.

$> unzip p21634633_212200_Linux-x86-64.zip
Archive:  p21634633_212200_Linux-x86-64.zip
   creating: dbserver_patch_21.210804/
  inflating: dbserver_patch_21.210804/cellboot_usb_pci_path
  inflating: dbserver_patch_21.210804/exadata.img.hw
  inflating: dbserver_patch_21.210804/README.txt
  inflating: dbserver_patch_21.210804/ExadataSendNotification.pm
  inflating: dbserver_patch_21.210804/patchmgr_functions
  inflating: dbserver_patch_21.210804/patchmgr
  inflating: dbserver_patch_21.210804/ExaXMLNode.pm
   creating: dbserver_patch_21.210804/linux.db.rpms/
  inflating: dbserver_patch_21.210804/exadata.img.env
  inflating: dbserver_patch_21.210804/dcli
 extracting: dbserver_patch_21.210804/dbnodeupdate.zip
  inflating: dbserver_patch_21.210804/ExadataImageNotification.pl
  inflating: dbserver_patch_21.210804/imageLogger
  inflating: dbserver_patch_21.210804/md5sum_files.lst
  inflating: dbserver_patch_21.210804/patchReport.py

If patchmgr is not updated, you might see a warning during its execution.

WARNING Your patchmgr is more than 60 days old. Check MOS 1553103.1 and make sure you have the most recent version.

Perform Exadata System Software Update

Preparation

Patchmgr requires an input file with the list of cluster nodes that are part of the patching.

$> vi /stage/db_nodes_list_file
vmcluster_node1
vmcluster_node2
vmcluster_node3
vmcluster_node4

When you run patchmgr on a separate system, ensure that this system can connect to the root user of all nodes using a passwordless SSH connection.

Run Precheck

Before performing the Exadata System Software Update, run a precheck.

$> cd /stage/dbserver_patch_21.210804
$>  patchmgr --dbnodes /stage/db_nodes_list_file \
      --precheck \
      --iso_repo /stage/20.1.12.0.0.210717/exadata_ol7_20.1.12.0.0.210717_Linux-x86-64.zip \
      --target_version 20.1.12.0.0.210717 \
      --allow_active_network_mounts \
      --nomodify_at_prereq \
      --log_dir /stage/logs

************************************************************************************************************
NOTE    patchmgr release: 21.210804 (always check MOS 1553103.1 for the latest release of dbserver.patch.zip)
NOTE
WARNING Do not interrupt the patchmgr session.
WARNING Do not resize the screen. It may disturb the screen layout.
WARNING Do not reboot database nodes during update or rollback.
WARNING Do not open logfiles in write mode and do not try to alter them.
************************************************************************************************************
2021-08-14 10:44:43 +0200        :INFO   : Checking hosts connectivity via ICMP/ping
2021-08-14 10:44:45 +0200        :INFO   : Hosts Reachable: [vmcluster_node1 vmcluster_node2 vmcluster_node3 vmcluster_node4]
2021-08-14 10:44:45 +0200        :INFO   : All hosts are reachable via ping/ICMP
2021-08-14 10:44:45 +0200        :Working: Verify SSH equivalence for the root user to node(s)
2021-08-14 10:44:45 +0200        :WARNING: patchmgr is launched from a remote/shared filesystem
2021-08-14 10:44:45 +0200        :WARNING: Ensure the filesystem is not hosted from any of the target nodes listed
2021-08-14 10:44:46 +0200        :INFO   : SSH equivalency verified to host vmcluster_node1
2021-08-14 10:44:47 +0200        :INFO   : SSH equivalency verified to host vmcluster_node2
2021-08-14 10:44:48 +0200        :INFO   : SSH equivalency verified to host vmcluster_node3
2021-08-14 10:44:49 +0200        :INFO   : SSH equivalency verified to host vmcluster_node4
2021-08-14 10:44:49 +0200        :SUCCESS: Verify SSH equivalence for the root user to node(s)
2021-08-14 10:44:57 +0200        :Working: Initiate precheck on 4 node(s)
2021-08-14 10:50:11 +0200        :Working: Check free space on node(s)
2021-08-14 10:50:39 +0200        :SUCCESS: Check free space on node(s)
2021-08-14 10:52:18 +0200        :Working: dbnodeupdate.sh running a precheck for the conventional update on node(s).

    SUMMARY OF WARNINGS AND KNOWN ISSUE(S) FOR vmcluster_node1:

    vmcluster_node1: # The following file lists the commands that would have been executed for removing rpms when specifying -M flag. #
    vmcluster_node1: # File: /var/log/cellos/nomodify_results.140821105221.sh. #
    vmcluster_node1: WARNING: Installed package verification prerequisite failed. See /var/log/cellos/dbnodeupdate_missing_package_files.140821105221.txt for details.
    vmcluster_node1: The following known issues will be checked for but require manual follow-up:
    vmcluster_node1: (*) - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12



    SUMMARY OF WARNINGS AND KNOWN ISSUE(S) FOR vmcluster_node2:

    vmcluster_node2: # The following file lists the commands that would have been executed for removing rpms when specifying -M flag. #
    vmcluster_node2: # File: /var/log/cellos/nomodify_results.140821105220.sh. #
    vmcluster_node2: WARNING: Installed package verification prerequisite failed. See /var/log/cellos/dbnodeupdate_missing_package_files.140821105220.txt for details.
    vmcluster_node2: The following known issues will be checked for but require manual follow-up:
    vmcluster_node2: (*) - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12



    SUMMARY OF WARNINGS AND KNOWN ISSUE(S) FOR vmcluster_node3:

    vmcluster_node3: # The following file lists the commands that would have been executed for removing rpms when specifying -M flag. #
    vmcluster_node3: # File: /var/log/cellos/nomodify_results.140821105221.sh. #
    vmcluster_node3: WARNING: Installed package verification prerequisite failed. See /var/log/cellos/dbnodeupdate_missing_package_files.140821105221.txt for details.
    vmcluster_node3: The following known issues will be checked for but require manual follow-up:
    vmcluster_node3: (*) - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12



    SUMMARY OF WARNINGS AND KNOWN ISSUE(S) FOR vmcluster_node4:

    vmcluster_node4: # The following file lists the commands that would have been executed for removing rpms when specifying -M flag. #
    vmcluster_node4: # File: /var/log/cellos/nomodify_results.140821105221.sh. #
    vmcluster_node4: WARNING: Installed package verification prerequisite failed. See /var/log/cellos/dbnodeupdate_missing_package_files.140821105221.txt for details.
    vmcluster_node4: The following known issues will be checked for but require manual follow-up:
    vmcluster_node4: (*) - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12


2021-08-14 10:56:57 +0200        :INFO   : SUMMARY FOR ALL NODES:
2021-08-14 10:56:23 +0200        :SUCCESS: vmcluster_node1 has state: SUCCESS WITH WARNING(S)
2021-08-14 10:56:28 +0200        :SUCCESS: vmcluster_node2 has state: SUCCESS WITH WARNING(S)
2021-08-14 10:56:33 +0200        :SUCCESS: vmcluster_node3 has state: SUCCESS WITH WARNING(S)
2021-08-14 10:56:37 +0200        :SUCCESS: vmcluster_node4 has state: SUCCESS WITH WARNING(S)
2021-08-14 10:56:57 +0200        :SUCCESS: Initiate precheck on node(s).
[INFO     ] Collected dbnodeupdate diag in file: Diag_patchmgr_dbnode_precheck_140821104441.tbz
-rw-rw-r-- 1 oracle oinstall 4188158 Aug 14 10:57 Diag_patchmgr_dbnode_precheck_140821104441.tbz
2021-08-14 10:57:01 +0200        :SUCCESS: Completed run of command: /stage/dbserver_patch_21.210804/patchmgr -dbnodes /stage/vmcluster1 -precheck -iso_repo /stage/20.1.12.0.0.210717/exadata_ol7_20.1.12.0.0.210717_Linux-x86-64.zip -target_version 20.1.12.0.0.210717 -allow_active_network_mounts -nomodify_at_prereq --log_dir /stage/vmcluster1_logs
2021-08-14 10:57:01 +0200        :INFO   : Precheck attempted on nodes in file /stage/vmcluster1: [vmcluster_node1 vmcluster_node2 vmcluster_node3 vmcluster_node4]
2021-08-14 10:57:01 +0200        :INFO   : Current image version on dbnode(s) is:
2021-08-14 10:57:01 +0200        :INFO   : vmcluster_node1: 20.1.9.0.0.210416
2021-08-14 10:57:01 +0200        :INFO   : vmcluster_node2: 20.1.9.0.0.210416
2021-08-14 10:57:01 +0200        :INFO   : vmcluster_node3: 20.1.9.0.0.210416
2021-08-14 10:57:01 +0200        :INFO   : vmcluster_node4: 20.1.9.0.0.210416
2021-08-14 10:57:01 +0200        :INFO   : For details, check the following files in /stage/logs:
2021-08-14 10:57:01 +0200        :INFO   :  - <dbnode_name>_dbnodeupdate.log
2021-08-14 10:57:01 +0200        :INFO   :  - patchmgr.log
2021-08-14 10:57:01 +0200        :INFO   :  - patchmgr.trc
2021-08-14 10:57:01 +0200        :INFO   : Exit status:0
2021-08-14 10:57:01 +0200        :INFO   : Exiting.

Run Update

If no issues were found during the precheck or all of them were fixed, start the update.

$> cd /stage/dbserver_patch_21.210804
$>  patchmgr --dbnodes /stage/db_nodes_list_file \
      --upgade\
      --iso_repo /stage/20.1.12.0.0.210717/exadata_ol7_20.1.12.0.0.210717_Linux-x86-64.zip \
      --target_version 20.1.12.0.0.210717 \
      --allow_active_network_mounts \
      --nomodify_at_prereq \
      --log_dir /stage/logs

************************************************************************************************************
NOTE    patchmgr release: 21.210804 (always check MOS 1553103.1 for the latest release of dbserver.patch.zip)
NOTE
WARNING Do not interrupt the patchmgr session.
WARNING Do not resize the screen. It may disturb the screen layout.
WARNING Do not reboot database nodes during update or rollback.
WARNING Do not open logfiles in write mode and do not try to alter them.
************************************************************************************************************
2021-08-14 11:08:40 +0200        :INFO   : Checking hosts connectivity via ICMP/ping
2021-08-14 11:08:41 +0200        :INFO   : Hosts Reachable: [vmcluster_node1 vmcluster_node2 vmcluster_node3 vmcluster_node4]
2021-08-14 11:08:41 +0200        :INFO   : All hosts are reachable via ping/ICMP
2021-08-14 11:08:41 +0200        :Working: Verify SSH equivalence for the root user to node(s)
2021-08-14 11:08:42 +0200        :WARNING: patchmgr is launched from a remote/shared filesystem
2021-08-14 11:08:42 +0200        :WARNING: Ensure the filesystem is not hosted from any of the target nodes listed
2021-08-14 11:08:43 +0200        :INFO   : SSH equivalency verified to host vmcluster_node1
2021-08-14 11:08:43 +0200        :INFO   : SSH equivalency verified to host vmcluster_node2
2021-08-14 11:08:44 +0200        :INFO   : SSH equivalency verified to host vmcluster_node3
2021-08-14 11:08:45 +0200        :INFO   : SSH equivalency verified to host vmcluster_node4
2021-08-14 11:08:46 +0200        :SUCCESS: Verify SSH equivalence for the root user to node(s)
2021-08-14 11:08:48 +0200        :Working: Initiate prepare steps on node(s).
2021-08-14 11:08:50 +0200        :Working: Check free space on node(s)
2021-08-14 11:09:17 +0200        :SUCCESS: Check free space on node(s)
2021-08-14 11:10:51 +0200        :SUCCESS: Initiate prepare steps on node(s).
2021-08-14 11:10:51 +0200        :Working: Initiate update on 4 node(s).
2021-08-14 11:10:52 +0200        :Working: dbnodeupdate.sh running a backup on 4 node(s).
2021-08-14 11:13:08 +0200        :SUCCESS: dbnodeupdate.sh running a backup on 4 node(s).
2021-08-14 11:13:08 +0200        :Working: Initiate update on node(s)
2021-08-14 11:13:08 +0200        :Working: Get information about any required OS upgrades from vmcluster_node1.
2021-08-14 11:13:08 +0200        :Working: Get information about any required OS upgrades from vmcluster_node2.
2021-08-14 11:13:08 +0200        :Working: Get information about any required OS upgrades from vmcluster_node3.
2021-08-14 11:13:08 +0200        :Working: Get information about any required OS upgrades from vmcluster_node4.
2021-08-14 11:13:19 +0200        :SUCCESS: Get information about any required OS upgrades from vmcluster_node3.
2021-08-14 11:13:19 +0200        :SUCCESS: Get information about any required OS upgrades from vmcluster_node1.
2021-08-14 11:13:19 +0200        :SUCCESS: Get information about any required OS upgrades from vmcluster_node2.
2021-08-14 11:13:19 +0200        :SUCCESS: Get information about any required OS upgrades from vmcluster_node4.
2021-08-14 11:13:20 +0200        :Working: dbnodeupdate.sh running an update step on vmcluster_node3.
2021-08-14 11:13:20 +0200        :Working: dbnodeupdate.sh running an update step on vmcluster_node2.
2021-08-14 11:13:21 +0200        :Working: dbnodeupdate.sh running an update step on vmcluster_node1.
2021-08-14 11:13:22 +0200        :Working: dbnodeupdate.sh running an update step on vmcluster_node4.
2021-08-14 11:25:36 +0200        :INFO   : vmcluster_node2 is ready to reboot.
2021-08-14 11:25:36 +0200        :SUCCESS: dbnodeupdate.sh running an update step on vmcluster_node2.
2021-08-14 11:25:41 +0200        :INFO   : vmcluster_node3 is ready to reboot.
2021-08-14 11:25:41 +0200        :SUCCESS: dbnodeupdate.sh running an update step on vmcluster_node3.
2021-08-14 11:25:42 +0200        :Working: Initiate reboot on vmcluster_node2.
2021-08-14 11:25:47 +0200        :Working: Initiate reboot on vmcluster_node3.
2021-08-14 11:25:49 +0200        :SUCCESS: Initiate reboot on vmcluster_node2.
2021-08-14 11:25:49 +0200        :Working: Waiting to ensure vmcluster_node2 is down before reboot.
2021-08-14 11:25:53 +0200        :SUCCESS: Initiate reboot on vmcluster_node3.
2021-08-14 11:25:53 +0200        :Working: Waiting to ensure vmcluster_node3 is down before reboot.
2021-08-14 11:25:57 +0200        :INFO   : vmcluster_node1 is ready to reboot.
2021-08-14 11:25:57 +0200        :SUCCESS: dbnodeupdate.sh running an update step on vmcluster_node1.
2021-08-14 11:26:03 +0200        :Working: Initiate reboot on vmcluster_node1.
2021-08-14 11:26:03 +0200        :INFO   : vmcluster_node4 is ready to reboot.
2021-08-14 11:26:04 +0200        :SUCCESS: dbnodeupdate.sh running an update step on vmcluster_node4.
2021-08-14 11:26:09 +0200        :Working: Initiate reboot on vmcluster_node4.
2021-08-14 11:26:10 +0200        :SUCCESS: Initiate reboot on vmcluster_node1.
2021-08-14 11:26:10 +0200        :Working: Waiting to ensure vmcluster_node1 is down before reboot.
2021-08-14 11:26:16 +0200        :SUCCESS: Initiate reboot on vmcluster_node4.
2021-08-14 11:26:16 +0200        :Working: Waiting to ensure vmcluster_node4 is down before reboot.
2021-08-14 11:28:31 +0200        :SUCCESS: Waiting to ensure vmcluster_node2 is down before reboot.
2021-08-14 11:28:31 +0200        :Working: Waiting to ensure vmcluster_node2 is up after reboot.
2021-08-14 11:28:37 +0200        :SUCCESS: Waiting to ensure vmcluster_node3 is down before reboot.
2021-08-14 11:28:37 +0200        :Working: Waiting to ensure vmcluster_node3 is up after reboot.
2021-08-14 11:28:53 +0200        :SUCCESS: Waiting to ensure vmcluster_node1 is down before reboot.
2021-08-14 11:28:53 +0200        :Working: Waiting to ensure vmcluster_node1 is up after reboot.
2021-08-14 11:29:00 +0200        :SUCCESS: Waiting to ensure vmcluster_node4 is down before reboot.
2021-08-14 11:29:00 +0200        :Working: Waiting to ensure vmcluster_node4 is up after reboot.
2021-08-14 11:29:14 +0200        :SUCCESS: Waiting to ensure vmcluster_node2 is up after reboot.
2021-08-14 11:29:14 +0200        :Working: Waiting to connect to vmcluster_node2 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:29:20 +0200        :SUCCESS: Waiting to ensure vmcluster_node3 is up after reboot.
2021-08-14 11:29:20 +0200        :Working: Waiting to connect to vmcluster_node3 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:29:36 +0200        :SUCCESS: Waiting to ensure vmcluster_node1 is up after reboot.
2021-08-14 11:29:36 +0200        :Working: Waiting to connect to vmcluster_node1 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:29:40 +0200        :SUCCESS: Waiting to connect to vmcluster_node2 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:29:40 +0200        :Working: Wait for vmcluster_node2 is ready for the completion step of update.
2021-08-14 11:29:43 +0200        :SUCCESS: Waiting to ensure vmcluster_node4 is up after reboot.
2021-08-14 11:29:43 +0200        :Working: Waiting to connect to vmcluster_node4 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:29:46 +0200        :SUCCESS: Waiting to connect to vmcluster_node3 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:29:46 +0200        :Working: Wait for vmcluster_node3 is ready for the completion step of update.
2021-08-14 11:30:03 +0200        :SUCCESS: Waiting to connect to vmcluster_node1 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:30:03 +0200        :Working: Wait for vmcluster_node1 is ready for the completion step of update.
2021-08-14 11:30:08 +0200        :SUCCESS: Waiting to connect to vmcluster_node4 with SSH. During Linux upgrades this can take some time.
2021-08-14 11:30:08 +0200        :Working: Wait for vmcluster_node4 is ready for the completion step of update.
2021-08-14 11:30:21 +0200        :SUCCESS: Wait for vmcluster_node2 is ready for the completion step of update.
2021-08-14 11:30:22 +0200        :Working: Initiate completion step from dbnodeupdate.sh on vmcluster_node2
2021-08-14 11:30:37 +0200        :SUCCESS: Wait for vmcluster_node3 is ready for the completion step of update.
2021-08-14 11:30:37 +0200        :Working: Initiate completion step from dbnodeupdate.sh on vmcluster_node3
2021-08-14 11:30:50 +0200        :SUCCESS: Wait for vmcluster_node4 is ready for the completion step of update.
2021-08-14 11:30:51 +0200        :Working: Initiate completion step from dbnodeupdate.sh on vmcluster_node4
2021-08-14 11:30:56 +0200        :SUCCESS: Wait for vmcluster_node1 is ready for the completion step of update.
2021-08-14 11:30:56 +0200        :Working: Initiate completion step from dbnodeupdate.sh on vmcluster_node1
2021-08-14 11:39:55 +0200        :SUCCESS: Initiate completion step from dbnodeupdate.sh on vmcluster_node2.
2021-08-14 11:40:18 +0200        :SUCCESS: Initiate completion step from dbnodeupdate.sh on vmcluster_node3.
2021-08-14 11:40:31 +0200        :SUCCESS: Initiate completion step from dbnodeupdate.sh on vmcluster_node4.
2021-08-14 11:40:43 +0200        :SUCCESS: Initiate completion step from dbnodeupdate.sh on vmcluster_node1.
2021-08-14 11:41:14 +0200        :SUCCESS: Initiate update on node(s).
2021-08-14 11:41:15 +0200        :SUCCESS: Initiate update on all node(s)
2021-08-14 11:41:15 +0200        :SUCCESS: Initiate update on 4 node(s).
2021-08-14 11:41:17 +0200        :SUCCESS: Completed run of command: /stage/20.1.12.0.0.210717/dbserver_patch_20.201211/patchmgr -dbnodes /stage/vmcluster1 -upgrade -iso_repo /stage/20.1.12.0.0.210717/exadata_ol7_20.1.12.0.0.210717_Linux-x86-64.zip -target_version 20.1.12.0.0.210717 -allow_active_network_mounts --log_dir /stage/vmcluster1_logs
2021-08-14 11:41:17 +0200        :INFO   : Upgrade attempted on nodes in file /stage/vmcluster1: [vmcluster_node1 vmcluster_node2 vmcluster_node3 vmcluster_node4]
2021-08-14 11:41:17 +0200        :INFO   : Current image version on dbnode(s) is:
2021-08-14 11:41:17 +0200        :INFO   : vmcluster_node1: 20.1.12.0.0.210717
2021-08-14 11:41:17 +0200        :INFO   : vmcluster_node2: 20.1.12.0.0.210717
2021-08-14 11:41:17 +0200        :INFO   : vmcluster_node3: 20.1.12.0.0.210717
2021-08-14 11:41:17 +0200        :INFO   : vmcluster_node4: 20.1.12.0.0.210717
2021-08-14 11:41:17 +0200        :INFO   : For details, check the following files in /stage/logs:
2021-08-14 11:41:17 +0200        :INFO   :  - <dbnode_name>_dbnodeupdate.log
2021-08-14 11:41:17 +0200        :INFO   :  - patchmgr.log
2021-08-14 11:41:17 +0200        :INFO   :  - patchmgr.trc
2021-08-14 11:41:17 +0200        :INFO   : Exit status:0
2021-08-14 11:41:17 +0200        :INFO   : Exiting.

References

8 comments

    1. Hi Davide,

      Oracle backs up all of the VM Clusters once a week (on Saturday as far as I know). My current customer is using TSM to create daily incremental backups. In the case of an error, we open an Oracle SR to request a restore of the VM. Afterwards we restore the incremental TSM backups for some directories.

      Patchmgr itself also creates a backup before the patching starts.

      Cheers
      Christian

      1. Christian — I believe you left out the VM OS image backup step. This step is noted in the Oracle doco for ExaCC VM OS Manual update:
        ./patchmgr -dbnodes dbs_group -backup -iso_repo zipped_iso_file -target_version target-version -allow_active_network_mounts

        Yes, it “may” have issues with NFS mount. As such, you can unmount the NFS mounts and re-run. We didn’t have issues with our NFS mounts (using the allow_active_network_mounts) until ExaCC on our X9’s Gen2.

        1. Hi Mark,

          I did not create a separate backup. As you can see from the output, patchmgr performs a implicit backup.
          2021-08-14 11:10:52 +0200 :Working: dbnodeupdate.sh running a backup on 4 node(s).
          2021-08-14 11:13:08 +0200 :SUCCESS: dbnodeupdate.sh running a backup on 4 node(s).
          Kind regards,
          Christian

  1. Hi,

    we got troubles with backup script (automatically executed as first step if you run update via OCI console ):

    troubles happen if NFS/Samba mounts visible – or LVM snaps existing,

    Script to run the backup manually:
    /opt/oracle.SupportTools/dbserver_backup.sh [–ignore-nfs-smbfs-mounts –check-rollback –get-backup-version ]

    Best to contact support in case it’s needed to run manually.

    1. Hi,

      do you also run into this issue when you run patchmgr manually with the parameter –allow_active_network_mounts? When we don’t use this parameter, we had also some troubles with NFS mounts.

      Cheers
      Christian

    2. Carlos:
      The documented command for ExaCC VM OS backup is ./patchmgr -dbnodes dbs_group -backup -iso_repo zipped_iso_file -target_version target-version -allow_active_network_mounts. The script you mentioned (dbserver_backup.sh) is called by that patchmgr (-backup) command. We were hesitant to use dbserver_backup.sh, regardless of supports recommendation, because it was unclear to use “what” we would be missing by not using the PATCHMGR cmd with the -backup flag. So, we opted to unmount the NFS mounts and then run the documented command.

Leave a Reply to Christian Gohmann Cancel reply

Your email address will not be published. Required fields are marked *