ExaCC: Upgrade Grid Infrastructure to 23ai using OCI CLI

In May 2024 Oracle announced the general availability of Oracle 23ai on ExaCC. In this blog post, I will upgrade an existing VM Cluster to 23ai using the OCI CLI.

If you want to use the OCI Web Console, please have a look at the blog post of Oracle ACE Director Fernando Simon: Upgrading to GI 23ai at ExaCC.


Exadata Software Version

Before you can upgrade an existing VM Cluster the following minimum Exadata Software Versions need to be installed.

  • Exadata Infrastructure: 23.1.x or higher
  • Exadata Guest VM: 23.1.8 or higher

ASM RDBMS Compatibility Version

23ai RACs only support database version 19c and higher. Databases with older versions must be terminated before the upgrade. The ASM database compatibility version must be set to, otherwise, the precheck will raise the following error.

Disk group RDBMS compatibility setting ...FAILED
DATAC3: PRVE-3180 : RDBMS compatibility for ASM disk group "DATAC3" is set to
        "", which is less than the minimum supported value

RECOC3: PRVE-3180 : RDBMS compatibility for ASM disk group "RECOC3" is set to
        "", which is less than the minimum supported value

To change the ASM Diskgroup attribute, connect to the ASM instance with SYSASM privilege.

SQL> SELECT name, compatibility, database_compatibility FROM v$asm_diskgroup;

---------- -------------------- --------------------



Before we can run the precheck, we need the OCID of the Grid Infrastructure Upgrade. Use the following OCI CLI call to retrieve all available updates of the type GI_UPGRADE.

$> oci db vm-cluster list-updates \
   --vm-cluster-id ocid1.vmcluster.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
   --update-type GI_UPGRADE \
   --query "data[].{\"01 Name\" : \"description\", \"02 Version\" : \"version\", \"03 OCID\" : \"id\", \"04 Actions\" : \"available-actions\"}" \
   --output table
| 01 Name                                           | 02 Version   | 03 OCID                                                                                        | 04 Actions                    | 05 Update  |
| Grid Infrastructure Release Update : | | ocid1.dbupdate.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | ['ROLLING_APPLY', 'PRECHECK'] | GI_UPGRADE |

In the next step run the precheck using the returned OCID.

$> oci db vm-cluster update \
   --vm-cluster-id ocid1.vmcluster.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
   --update-action PRECHECK \
   --update-id ocid1.dbupdate.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

The easiest way to monitor the initiated work request is to log in to the OCI Web Console and navigate to the VM Cluster overview page.

Work Request for the Precheck


After the precheck is successful, the upgrade can be started with the following command.

$> oci db vm-cluster update \
   --vm-cluster-id ocid1.vmcluster.oc1.eu-frankfurt-1.abtheljrphmoulymvecagl6bf5gd2d7yaoc6kbqxrbpnfmgse7owjucwc6ba \
   --update-action ROLLING_APPLY \
   --update-id ocid1.dbupdate.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Again, monitor the work request in the OCI Web Console.

Work Request for the Upgrade


Log Files

The log files of the work requests are created on the VM Cluster Nodes in the directory /var/opt/oracle/log/grid/upgrade.

$> ls -ltr /var/opt/oracle/log/grid/upgrade/
total 7192
-rw-r----- 1 oracle oinstall 1057448 Jun  4 14:41 pilot_2024-06-04_02-37-13-PM_10502
-rw-r----- 1 oracle oinstall   52634 Jun  4 14:41 dbaastools_2024-06-04_02-37-07-PM_10055.log
-rw-r----- 1 oracle oinstall 2785169 Jun  4 15:12 pilot_2024-06-04_02-43-54-PM_51097
-rw-r----- 1 oracle oinstall   70354 Jun  4 15:12 dbaastools_2024-06-04_02-43-49-PM_50728.log


To run the CVU we need to download the 23ai image. You can download the image with dbaascli. In the first step identify the available images.

$> dbaascli cswLib showImages --product GRID
DBAAS CLI version
Executing command cswLib showImages --product GRID
Job id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Session log: /var/opt/oracle/log/cswLib/showImages/dbaastools_2024-06-04_12-34-46-PM_306641.log
Log file location: /var/opt/oracle/log/cswLib/showImages/dbaastools_2024-06-04_12-34-46-PM_306641.log

############ List of Available grid Artifacts  #############

  DESCRIPTION=19c JAN 2024 GI Image
  DESCRIPTION=19c APR 2024 GI Image
  DESCRIPTION=23ai grid image
  DESCRIPTION=19c JUL 2023 GI Image
  DESCRIPTION=19c OCT 2023 GI Image
Images can be downloaded using their image tags. For details, see help using 'dbaascli cswlib download --help'.
dbaascli execution completed

To download the image use the IMAGE_TAG.

$> dbaascli cswLib download --imageTag grid_23.
DBAAS CLI version
Executing command cswLib download --imageTag grid_23.
Job id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Session log: /var/opt/oracle/log/cswLib/download/dbaastools_2024-06-04_12-35-43-PM_309874.log
Image location=/var/opt/oracle/dbaas_acfs/gi_images/grid_23.
Download succeeded
Log file location: /var/opt/oracle/log/cswLib/download/dbaastools_2024-06-04_12-35-43-PM_309874.log
dbaascli execution completed

$> ls -l /var/opt/oracle/dbaas_acfs/gi_images/grid_23.
total 1021956
-rwx------ 1 grid oinstall 1046182666 Jun  4 09:17 grid_23.

Now you can unzip the file and run the Cluster Verification Utility (CVU or cluvfy) to verify if your system is ready for the upgrade.

# Run as root user
$> mkdir /u02/grid
$> chown grid:oinstall /u02/grid
$> su - grid

# Run as grid user
$> unzip -q -d /u02/grid /var/opt/oracle/dbaas_acfs/gi_images/grid_23.
$> /u02/grid/runcluvfy.sh stage -pre crsinst -upgrade -src_crshome /u01/app/ -dest_crshome /u02/grid -dest_version -rolling

Pre-check for cluster services setup was successful.

Warnings were encountered during execution of CVU verification request "stage -pre crsinst".

RPM Package Manager database ...INFORMATION
PRVG-11250 : The check "RPM Package Manager database" was not performed because
it needs 'root' user privileges.

Refer to My Oracle Support notes "2548970.1" for more details regarding errors

Kernel crash dumping mechanism ...WARNING
oranode001: PRVE-10329 : Kernel crash dumping mechanism was not active on
                 node "oranode001". Command "/bin/systemctl is-active
                 kdump.service" returned "failed"

oranode002: PRVE-10329 : Kernel crash dumping mechanism was not active on
                 node "oranode002". Command "/bin/systemctl is-active
                 kdump.service" returned "failed"

CVU operation performed:      stage -pre crsinst
Date:                         Jun 4, 2024, 2:03:33 PM
CVU version:                  Standalone (042324x8664)
Clusterware version:
CVU home:                     /u02/grid
Grid home:                    /u01/app/
User:                         grid
Operating system:             Linux5.4.17-2136.322.6.4.el8uek.x86_64


Oracle Home

The Oracle Home location changed from /u01/app/ to /u02/app/ Oracle reduced the size of the Oracle Home drastically.

$> du -sh /u02/app/
3.1G    /u02/app/

The old Oracle Home /u01/app/ still exists and will not be cleaned up during the upgrade.

$> du -sh /u01/app/
12G     /u01/app/


Leave a Reply

Your email address will not be published. Required fields are marked *