Exadata Storage Cell Critical problem : RS-7445 [Serv MS leaking memory] [It will be restarted]


Known defect:
Bug 19790644 - RS-7445 [SERV MS LEAKING MEMORY]
Issue is a memory leak in the Java executable.
This bug affects systems running with JDK 7u51 or later versions (1.7.0_55-b13) which are 11.2.3.3.1 and 12.1.1.1.1
Systems running 11.2.3.3.0 or 12.1.1.1.0 are not affected as they use 1.7.0_25-b15
Solution:
1: Error is ignorable as MS service will be re-started automatically.
2: While patching the storage cell with one-off patches is not generally recommended, if there are issues where the MS service is not automatically re-started, the JDK needs upgraded on the Storage Cell
Use Patch 20328167: TRACKING BUG FOR JDK 1.7.0.72- B33 PATCH (wrapper for 20328167: Oracle JDK 7 Update 72 b33 or later)
3: If no other issues are being seen, the recommended action is to wait for future release of Exadata Software Version that will contain the fix (actual version is to be determined).
TABLE OF CONTENTS
-----------------
a) Contents of zip file
b) Installing this one-off patch on Exadata Storage Servers
c) Rolling back this one-off patch on Exadata Storage Servers
a) Contents of zip file
-----------------------
The zip file contains one rpm and this README.txt.
The rpms is:
jdk-7u72-linux-x64.rpm
This patch fixes:
BUG 19790644 - RS-7445 [SERV MS LEAKING MEMORY]
Detail about jdk patch:
BUG 20090815 - Oracle JDK 7 Update 72 b33
b) Installing this one-off patch on Exadata Storage Servers
-----------------------------------------------------------
Download this patch p20359157_12.1.1.1.1_Linux-X64.zip to a anarnode 
and then unzip this patch to copy jdk rpm to the cells.
b.1) unzip the file p20359157_12.1.1.1.1_Linux-X64.zip
[root@anarnode ~]# cp jdk-7u72-linux-x64.rpm -d /tmp
b.2) You can use dcli to copy the zip file. 
The following command copies the p20090815_17072_Linux-x86-64.zip
[root@anarnode ~]# dcli -l root -g cell_group -f /tmp/jdk-7u72-linux-x64.rpm -d /tmp
b.3) Apply jdk patch after MS shutdown. java version should become "1.7.0_72"
[root@anarnode ~]# dcli -l root -g cell_group cellcli -e alter cell shutdown services ms
cellnode: 
cellnode: Stopping MS services...
cellnode: The SHUTDOWN of MS services was successful.
...
[root@anarnode ~]# dcli -l root -g cell_group rpm -i /tmp/jdk-7u72-linux-x64.rpm
cellnode: Unpacking JAR files...
cellnode: rt.jar...
cellnode: jsse.jar...
cellnode: charsets.jar...
cellnode: tools.jar...
cellnode: localedata.jar...
cellnode: jfxrt.jar...
...
Verify upgraded java version by following command.
[root@anarnode ~]# dcli -l root -g cell_group java -version
cellnode: java version "1.7.0_72"
cellnode: Java(TM) SE Runtime Environment (build 1.7.0_72-b33)
cellnode: Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)
...
b.3) Restart MS
[root@anarnode ~]# dcli -l root -g cell_group cellcli -e alter cell startup services ms
cellnode: 
cellnode: Starting MS services...
cellnode: The STARTUP of MS services was successful.
...
b.4) Cleanup patch file 
[root@anarnode ~]# dcli -l root -g cell_group cellcli rm /tmp/jdk-7u72-linux-x64.rpm
c) Rolling back this one-off patch on Exadata Storage Servers
-------------------------------------------------------------
After the patch installation, jdk-1.7.0_55-fcs and jdk-1.7.0_72-fcs are
installed for each cells. 
[root@anarnode ~]# dcli -l root -g cell_group rpm -q jdk-1.7.0_55-fcs
cellnode: jdk-1.7.0_55-fcs
...
[root@anarnode ~]# dcli -l root -g cell_group rpm -q jdk-1.7.0_72-fcs
cellnode: jdk-1.7.0_72-fcs
...
To rollback jdk-1.7.0_72, uninstall jdk-1.7.0_72-fcs and jdk-1.7.0_55-fcs becomes active version.
[root@anarnode ~]# dcli -l root -g cell_group cellcli -e alter cell shutdown services ms
...
[root@anarnode ~]# dcli -l root -g cell_group rpm -e jdk-1.7.0_72-fcs
...
[root@anarnode ~]# dcli -l root -g cell_group java -version
cellnode: java version "1.7.0_55"
cellnode: Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
cellnode: Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)
...
[root@anarnode ~]# dcli -l root -g cell_group cellcli -e alter cell startup services ms
...
More information: Exadata Storage Cell reports error RS-7445 [Serv MS Leaking Memory] (Doc ID 1954357.1)
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: