Latency and SocketTimeoutException Between OSB And Back End Services Causing Long Response Times of Proxy Services

/ Lakshman

Symptoms

You have been observing very slow response from your proxy service that is making a call to another server.  The low performance of this service leads to incomining messages beeing queued.

In addition, the following was observed in the OSB logs that appear related to the latency:
<Error> <WliSbTransports> [ACTIVE] ExecuteThread: ’11’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <> <BEA-381304> <Exception in HttpOutboundMessageContext.RetrieveHttpResponseWork.run: java.net.SocketTimeoutException
java.net.SocketTimeoutException
at weblogic.net.http.AsyncResponseHandler$MuxableSocketHTTPAsyncResponse$SocketTimeoutNotification.<clinit>(AsyncResponseHandler.java:551)
at weblogic.net.http.AsyncResponseHandler$MuxableSocketHTTPAsyncResponse.handleTimeout(AsyncResponseHandler.java:396)
at weblogic.net.http.AsyncResponseHandler$MuxableSocketHTTPAsyncResponse.timeout(AsyncResponseHandler.java:502)
at weblogic.socket.SocketMuxer$TimerListenerImpl.timerExpired(SocketMuxer.java:994)
at weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

Cause

The cause of the latency of the proxy services have been identified in the load balancer settings. The load balancer was configured to take a non-responsive server out of its pool for 5 minutes.

Solution

To change this, modify the load balancer settings, change the time unresponsive managed servers are taken out of the pool of available servers from 5 minutes to 30 seconds.  This will allow enough time for a busy managed server to recover but will not take it out of the pool for too long. Once the load balancer settings were changed, the issue was no longer observed.

Leave a comment