In the last few months I was struggeling with performance issues during the installation of different Database Proactive Bundle Patches on Solaris SPARC. On the server, the OPatch version 220.127.116.11.11 (October 2017) was installed. Now with the release of OPatch 18.104.22.168.12 (January 2018) the installation time decreases from 3,5 to 0,5 hours.
What was the Problem?
First of all, the affected servers were very I/O bonded. So every file I/O took more time and should be avoided as much as possible. In November I installed the October 2017 Bundle Patch for several Oracle 12c Release 1 installations running on Solaris SPARC (64-bit).
After I started OPatch (version 22.214.171.124.11), the execution hung at the point “Backing up files…” for 2,5 hours. To investigate the problem, I enabled the Debug mode.
$> export OPATCH_DEBUG=true
From the output, I could see, that the libserver12.a file was copied more than 1000 times!
ArchiveAction::restorable() Action details: src = "/app/oracle/product/126.96.36.199b/lib/libserver12.a", dst = "/app/oracle/product/188.8.131.52b/.patch_storage/NApply/2018-01-25_12-45-27PM/backup/lib/libserver12.a" ArchiveAction::backupForRestore() ArchiveAction::backupForRestore(): copy file from "/app/oracle/product/184.108.40.206b/lib/libserver12.a" to "/app/oracle/product/220.127.116.11b/.patch_storage/NApply/2018-01-25_12-45-27PM/backup/lib/libserver12.a" SystemCall::backupFile() SystemCall::backupFile(): back up file /app/oracle/product/18.104.22.168b/lib/libserver12.a to /app/oracle/product/22.214.171.124b/.patch_storage/NApply/2018-01-25_12-45-27PM/backup/lib/libserver12.a /app/oracle/product/126.96.36.199b/lib/libserver12.a
On the 24th of January, OPatch 188.8.131.52.12 was released and I used this version to install the January 2018 Bundle Patch. After 30 minutes, the installation was finished – this was a big surprise!
For further investigation I installed the Bundle Patch in Debug Mode using the old and new version of OPatch and compared the output. Now the libserver12.a file was only copied once!
$> grep -c "back up file /u00/app/oracle/product/184.108.40.206/lib/libserver12.a" opatch_1220111.log 1087 $> grep -c "back up file /u00/app/oracle/product/220.127.116.11/lib/libserver12.a" opatch_1220112.log 1
I have two recommendation for OPatch (at least for such situations).
- Download the latest version from My Oracle Support (Patch No. 6880880) before installing a new patch
- Save older versions of OPatch
Why should I keep older versions of OPatch? Because older versions are not downloadable and are only accessible by opening a Service Request. When you now encounter a problem with the current version of OPatch, you are able to switch back to your “working” version of OPatch – except you need the new version for installing the patch.