Using IBM HTTP Server diagnostic capabilities with WebSphere

Notes
The ?showmodule parameter to the server status report requires one of the following levels of IBM HTTP Server:
  • 2.0.42.x and 2.0.47.x with cumulative e-fix PK01070 or later
  • 1.3.26.x and 1.3.28.x with cumulative e-fix PK05084 or later
  • 6.0.x with fix pack 6.0.2 or later
  • 6.1 or later

Current maintenance for your release of IBM HTTP Server can be accessed from the Recommended Updates for IBM HTTP Server page.

IBM HTTP Server 6.1 and later users should refer to this page instead.

Problems occasionally encountered with WebSphere applications include

When WebSphere requests are routed through IBM HTTP Server using the WebSphere plug-in, the same symptoms may have different causes, such as:

Understanding and using IBM HTTP Server serviceability features can reduce the time required to locate the cause of a problem for requests routed through IBM HTTP Server to WebSphere Application Server.

Preparation

Before a problem occurs, enable these IBM HTTP Server features.

Enable mod_status and ExtendedStatus

mod_status provides a report of current web server activity. This includes the current set of requests which are being processed, the length of time that the request has been active, and the active plug-in module (if any) which is handling the request.

In order to see details about each request, the ExtendedStatus directive must be set to On in httpd.conf.

Add response time to the access log

When there are concerns about page display time, it is important to know which types of requests are responded to slowly. By recording the response time for every request in the access log, analysis of the access log can provide information about where the problem may lie.

The response time in seconds can be added to the end of every line of the access log by adding %T to the end of the format string used for logging. This format string is configured using the LogFormat directive in httpd.conf.

In the following example, a new format called responsetime has been created with the response time in seconds at the end of the line, and the access log uses that format.

LogFormat "%h %l %u %t \"%r\" %>s %b %T" responsetime
...
CustomLog logs/access_log responsetime

Note:Any programs used in your environment to analyze the access log may be impacted by adding this additional field. If there are such programs in use, investigate their requirements before implementing the change.

The LogFormat and CustomLog directives are described in the mod_log_config documentation.

Diagnosing unresponsive behavior

In order to collect documentation from the appropriate component, it is important to determine if the unresponsive behavior is caused by an issue within IBM HTTP Server or within WebSphere. The easiest way to check this is to connect directly to the WebSphere port with the browser and run the application in that manner.

If the application doesn't respond normally when connecting directly to WebSphere, then the problem is not expected to involve IBM HTTP Server or the WebSphere plug-in. Consult the WebSphere MustGather documents to collect the appropriate documentation.

If the application responds normally when connecting directly to WebSphere but the application does not respond when the application request is routed through IBM HTTP Server, then the problem is not expected to involve WebSphere.

The next step is to view the server status page, including the optional module display. The URL will be http://www.example.com/server-status/?showmodule. Substitute your own hostname for www.example.com.

The bottom portion of the report will look something like this:

SrvPIDAccMModuleCPU SSReqConnChildSlotClientVHostRequest
0-044610/136/136_ 0.11000.02.332.33 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/135/135Wmod_was_ap20_http.c 0.03300.02.312.31 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/135/135Wmod_was_ap20_http.c 0.15300.02.312.31 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/136/136_ 0.11000.02.342.34 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/134/134_ 0.15000.02.312.31 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/135/135_ 0.12000.02.332.33 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/133/133Wmod_was_ap20_http.c 0.10300.02.292.29 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1
0-044610/135/135Wmod_was_ap20_http.c 0.09300.02.332.33 127.0.0.1localhost.localdomainGET /snoop/ HTTP/1.1

Ignore entries with _ in the M column, since those IHS threads are idle. The SS column contains the number of seconds since the start of the request.

For requests with an unexpectedly long time in the SS column, check the value in the Module column. If the module is mod_was_ap20_http.c or mod_app_server_http.c, then the request is currently being processed by the WebSphere plug-in. Check the WebSphere plug-in log file for processing problems. If you were unable to determine if the application responds normally when connecting directly to WebSphere, perhaps due to application authentication requirements which are fulfilled by modules enabled in IBM HTTP Server, then the issue is most likely in WebSphere.

If the module which is processing the stalled request is a third-party module, such as one that handles authentication, consult the vendor of that module for instructions on diagnosing the problem.

Otherwise, follow IBM HTTP Server MustGather instructions for web server hangs. (Unix and Linux systems) (Windows systems)

Diagnosing slow response times

A common cause of slow response times for all types of requests is an insufficient number of IBM HTTP Server worker threads configured in httpd.conf. During peak loads, requests may be queued for some time until a worker thread becomes available to handle the request. This tuning is described in the performance tuning guide.

If there is not a problem with the number of configured worker threads, check the response times in the access log to see which requests are responded to slowly. Here are some possible conclusions: