Posts Tagged ‘Solaris 10’

How To Change Java Heap Size In R12 To Avoid Java.Lang.OutOfMemoryError: Java Heap Space Error?

December 24th, 2017, posted in Oracle, Solaris

Recently we had encountered a situation, wherein end users where not able to get the R12 login page and
it just hangs, but all the opmn processess were up and running – status gives status as
alive for all the components viz., HTTP, oafm, oacore & forms. The same environment was available and
end users were able to access without any isuses 15 minutes before. This error happens only when
multiple people login to the same page and perform simillar activity like employee self service form
etc. So where and what exactly could be the problem.

This is how we approached and resolved the issue.

First we checked apache error log. The following error was reported in the error log.

[warn] [client IP Address ] oc4j_socket_recvfull timed out
[error] [client IP Address ] mod_oc4j: There is no oc4j process (for destination: application://oacore) available to service request.
[error] [client  IP Address] mod_oc4j: request to OC4J [servername:OC4J AJP Port Range for Oacore]
failed: Connect failed

Above error message does give us a hint that the problem is with oacore, but as i said earlier the oacore is alive according to opmn.

Next, we verified the oacore log file.


Error Message in the file

Exception in thread “OC4JMonitorThread” java.lang.OutOfMemoryError: Java heap space

While checking the error in file, parallely we opened another window to see the CPU and memory usage and we could see one java process was taking more than 100% CPU. This process was spawned by the opmn -d process and the process id didnt match with the oacore process id. Hint: status will give the status as well the process id. It looked like a end client java process.

At this stage we had 3 options.

1. Kill the java process which is consuming high cpu.
2. Bounce oacore component
3. Adjust java (jvm) memory parameters

Each of the above options has its own implications and advantages. To minimize the downtime we decided  to kill the java processes, and the moment we killed the java process, all the browsers which were hanging reported instantaneously Internal Server Error. This proved to be a bad decision.

So we moved to the next option, bouncing the oacore service which we are sure will resolve the issue  (temporarily) and it did as expected. Basically when you bounce the services all the existing
connections and processess will be released which results in more available memory when you re-start the services.

Ok, now we tackled the problem and provided a temporary solution but we need to find a long term solution. This is option 3, adjusting java memory size.

Steps to change the heap size.

First, you need to identify how much is the maximum heap size that you can set. Click here.

Once you had identified the maximum heap size, we need to change the configuration files to make it

Step 1: Edit opmn.xml file.

Location : $INST_TOP/ora/10.1.3/opmn/conf/

Open opmn.xml

Search for string Xms or Xmx or module-id=”OC4J”

This search should lead you to below location

‘<‘process-type id=”oacore” module-id=”OC4J” status=”enabled” working-dir=”$ORACLE_HOME/j2ee/home”‘>’
‘<‘category id=”start-parameters”‘>’
‘<‘data id=”java-options” value=”-server -verbose:gc -Xmx512M -Xms128M ……]

The default value for Maximum (-Xmx) and Minimum (-Xms) heap sizes are 512M and 128M respectively.

Again here you have options, you can set both Xms and Xmx has the same value as Xmx if you feel all your sessoins require higher memory or set a lower value for Xms and the maximum value for Xmx. Dont forget to change the values under ‘<‘category id=”stop-parameters”‘>’

opmn.xml also contains jvm configurations for other components – oafm & forms.

Step 2: Edit file.

Location : ($INST_TOP/ora/10.1.3/j2ee/oacore/config)

This step is optional since we had already made changes in opmn.xml but there is no harm in making the change here. This step will come handy for troubleshooting specific components of Oracle viz., configurator, iSupplier or any other option which heavily utilizes/consumes CPU/memory.

Search for string Xms or Xmx or wrapper.

Option 1: If you find any of the above parameters change the values corresponding to the value you had  mentioned in opmn.xml or even more than that, as long as you dont exceed the maximum heap size limit.

Option 2: If you DO NOT find any of the above parameters, then make an entry like this, under the heading “# Java Object Cache Configuration Parameters”

wrapper.bin.parameters=-Xms[Value]M -Xmx[Value]M -XX:NewSize=256M -XX:MaxNewSize=256M

Step 3: Edit Applications Context file


Location: $INST_TOP/appl/admin/SID_hostname.xml

search for string s_oacore_jvm_start_options

Change Xms and Xmx value. Repeat the same step for parameter s_oacore_jvm_stop_options.

Changes made in Step 3 will take effect the next time you run autoconfig, whereas Step 1 & 2 changes will take effect the next time you bounce opmn services, but the values are not permanent in the sense these will be wiped off next time you run autoconfig. Yes you can preserve the changes by placing it inbetween Begin and End customizations.

You can also increase the Garbage Collection threads parameter (-XX:ParallelGCThreads) to a higher value if you have JDK 1.5 or more than 2 cpus or more memory. For more information on this you can refer to Metalink Note: 362851 – Guidelines to setup the JVM in Apps Ebusiness Suite 11i and R12.


oc4j_socket_recvfull Timed Out

December 20th, 2017, posted in Oracle, Solaris
[Wed Nov 1 11:37:28 2016][warn] [client] oc4j_socket_recvfull timed out
[Wed Nov 1 11:37:28 2016] [error] [client] [ecid: 1509525146:,0] mod_oc4j:
request to OC4J failed: recv failed (errno=4)


Troubleshooting MOD_OC4J_0080 MOD_OC4J_0058 MOD_OC4J_0035 MOD_OC4J_0121 MOD_OC4J_0013 Errors In the HTTP Server error_log file [ID 329456.1]

The MOD_OC4J* errors basically convey that the HTTP Server (through mod_oc4j) is unable to communicate with the OC4J instance in question. Some of the possible causes are :

  • 1. Internet Explorer (KeepAlive) Bug
  • 2. Firewalls
  • 3. Connection timeouts between HTTP Server/mod_oc4j & OC4J
  • 4. OC4J instance running short on memory
  • 5. OC4J JVMs loaded
  • 6. Application(s) blocking up OC4J

Connection timeouts between HTTP Server/mod_oc4j & OC4J

In $ORACLE_HOME/Apache/Apache/conf/httpd.conf, increase the Timeout directive value to, say, 600 seconds. (This should be smaller than the Firewall timeout setting, if any)

In $ORACLE_HOME/Apache/Apache/conf/mod_oc4j.conf below the tag add the following:

Oc4jCacheSize 0

Oc4jConnTimeout 600 (This is again in seconds &, again, should be smaller than the Firewall timeout setting, if any)

Update the configuration as follows:

$ORACLE_HOME/dcm/bin/dcmctl updateconfig -ct ohs
$ORACLE_HOME/opmn/bin/opmnctl restartproc process-type=HTTP_Server


Out Of Memory Not Enough Space Solaris 11

December 17th, 2017, posted in Oracle, Solaris
SQL> startup mount ;Out Of Memory Not Enough Space Solaris 11,Out Of Memory , Not Enough Space Solaris 11,Solaris 11,Solaris 10,Oracle 12c on Solaris 11.3,Oracle 12c,Solaris 11.3,
ORA-27102: out of memory
SVR4 Error: 12: Not enough space
Additional information: 1671
Additional information: 16106127360
Additional information: 64424509440


The Above error appeared When trying to start Oracle 12c on Solaris 11.3, this is published bug
Solaris using something called OSM , Optimized Shared memory You can know more about it here 


To avoid this error as workaround just set the parameter

Check OS BIT is 32 or 64 For Solaris and Linux

May 15th, 2016, posted in Solaris

Is my Operating System 64-bit?

In Solaris, from the command line (you don’t have to be root in most cases) run this command:
/usr/bin/isainfo -kv

If your OS is 64-bit, you will see output like:
64-bit sparcv9 kernel modules

If your OS is 32-bit, you will get this output:
32-bit sparc kernel modules

For Linux users :

If you are running Linux, you can check your distribution with the uname command:

uname -m

The output will read x86_64 for 64-bit and i686 or similar for 32-bit.

How about this Oracle install? Is it 64-bit?

The question here is weather your Oracle binaries are 64-bit. While some of the binaries associated with Oracle may be 32-bit, the important ones will be 64 bit. To check those, follow these steps from the command line:

file oracl*

This will display the file type of your oracle binaries. If you are running 64-bit binaries, the output should look like this:

oracle: ELF 64-bit MSB executable SPARCV9 Version 1, dynamically linked, not stripped
oracleO: ELF 64-bit MSB executable SPARCV9 Version 1, dynamically linked, not stripped

If your binaries are 32-bit, the output will look like this:

oracle: ELF 32-bit MSB executable SPARC Version 1, dynamically linked, not stripped


Taking explorer on SUN machines

April 21st, 2016, posted in Linux OS

If your system is a SunFire midframe server (38×0, 48×0, or 68×0 model), use this command to include data from the system console:

/opt/SUNWexplo/bin/explorer -q -e -w default,scextended

If your system uses an alom (advanced lights out manager), some additional data can be collected. Systems that use that include:
Sun Fire V210,V240,V250,V245,V440,V440R,V445 Server Netra 240,440 (AC) Server
Use this command:

# /opt/SUNWexplo/bin/explorer -q -e -w default,alomextended


If your system uses T1 processors (system type starts with T, and uname -a command will say architecture is sun4v), use this command:

# /opt/SUNWexplo/bin/explorer -q -e -w default,Tx000

For any other model of Sun system, use this command:

# /opt/SUNWexplo/bin/explorer -q -e


The -q option suppresses printing of error messages during the run.
The -e option prevents explorer from automatically sending email.
The -w scextended option collects data from the system console (sc).

If you cannot run explorer for some reason, here are the most important things to collect:

/var/adm/messages or messages.# file for the time when the crash was taken. Be sure to include any messages that occurred just before panic.

/etc/system file

output of the showrev -p command, for a list of patches on system

Detailed hardware configuration of the system. To determine which command to use, issue the command uname -m to find the system architecture type.

If it is sun4u, send the output of this command:
/usr/platform/sun4u/sbin/prtdiag -v
If it is sun4v, send the output of this command:
/usr/platform/sun4v/sbin/prtdiag -v
Output of this command:
prtconf -vp

If your system is a SunFire midframe server (38×0, 48×0, or 68×0 model), include the output of these commands to collect data from the system console:

showlogs -v
showlogs -d showboards -v showplatform -v showenvironment -v showsc -v

Link :