Tomcat 3.3.1a AIX 5.3 and IBM JDK 1.3.1 - Tomcat Hangs

["Himanshu Thube" <thube.himanshu (at) gmail.com>]

Hello All,

We have a problem in our production system. Following is our environment.

Tomcat 3.3.1a
AIX 5.3
IBM JDK 1.3.1
Apache/1.3.28

This system was working fine previously without any problems for 3 years the
only change that we did was changed from AIX 5.1 ML 4 to AIX 5.3 ML5 and
below problems started.

Tomcat process consumes more than 90% of CPU and finally hangs. This is
observed sometimes 5 times a day and we have to restart all
servers. Sometimes it works fine for a day or two. Tried to put all logs on
debug, but no errors anywhere. While starting Tomcat we also redirected the
errors and standard output to a log but no errors there as well. We tried to
check for any user behaviour or specific request that might be causing this
but nothing was found.

Only exceptions that we get are :

IOException in R

java.io.IOException: There is no process to read data written to a pipe.
at java.net.SocketOutputStream.socketWrite(Native Method)
at java.net.SocketOutputStream.write(SocketOutputStream.java(Compiled
Code))
at org.apache.tomcat.modules.server.HttpResponse.doWrite(
Http10Interceptor.java(Compiled Code))
at org.apache.tomcat.modules.server.HttpResponse.doWrite(
Http10Interceptor.java(Compiled Code))
at org.apache.tomcat.core.OutputBuffer.realWriteBytes
(OutputBuffer.java(Compiled
Code))
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java(Compiled
Code))
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java(Compiled
Code))
at org.apache.tomcat.util.buf.IntermediateOutputStream.write (
C2BConverter.java(Compiled Code))
at java.io.OutputStreamWriter.write(OutputStreamWriter.java(Compiled Code))
at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java(Compiled
Code))
at org.apache.tomcat.util.buf.CharChunk.append (CharChunk.java(Compiled
Code))
at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java(Compiled
Code))
at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java(Compiled
Code))
at org.apache.tomcat.util.buf.CharChunk.append (CharChunk.java(Compiled
Code))
at org.apache.tomcat.core.OutputBuffer.write(OutputBuffer.java:274)
at org.apache.tomcat.modules.generators.ExceptionHandler.doService(
ErrorHandler.java:658)
at org.apache.tomcat.core.Handler.service (Handler.java(Compiled Code))
at org.apache.tomcat.core.Handler.service(Handler.java(Compiled Code))
at org.apache.tomcat.modules.generators.ErrorHandler.handleErrorImpl(
ErrorHandler.java:414)
at org.apache.tomcat.modules.generators.ErrorHandler.handleError (
ErrorHandler.java:184)
at org.apache.tomcat.core.ContextManager.handleError(ContextManager.java(Compiled
Code))
at org.apache.tomcat.core.ContextManager.service(ContextManager.java(Compiled
Code))
at org.apache.tomcat.modules.server.Http10Interceptor.processConnection (
Http10Interceptor.java:176)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java(Compiled
Code))
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:516)
at java.lang.Thread.run (Thread.java:512)



However this exception does not cause Tomcat to hang because the time when
exception is thrown and Tomcat hangs has never matched even with a
difference of hour.

Can someone please help ? Changing anything in the environment would be very
very difficult. Can someone atleast tell how we can atleast get what is the
error that is causing this in Tomcat ?

--
Himanshu