The default semaphore configuration on Solaris is not adequate for using Apache/IHS with SysV semaphores for the accept mutex for more than a handful of child processes. The problem is that the web server uses a kernel "undo structure" for each child process waiting on the accept mutex, and the Solaris kernel allocates a relatively small number of these structures by default. By design, the web server does not use SysV semaphores by default on Solaris, but there are some situations where the administrator may be advised to use SysV semaphores.
A complicating issue with the semaphore tuning is that there may
be other applications running on the system that use semaphore
resources. The description here merely suggests what is necessary
for the web server's accept mutex. You may need additional kernel
resources for other applications. Thus, it is suggested that you
allocate plenty of extra kernel semaphore resources because they
can't be changed dynamically but instead require editing
/etc/system
and rebooting the system.
With Apache/IHS 1.3, one of the symptoms of an inadequate kernel SysV semaphore configuration is a message like this in the error log:
accept_mutex_on: No space left on device(This is the ENOSPC error.) The Apache parent process will exit shortly thereafter.
The active kernel semaphore resources are displayed by the
sysdef
command. Here is the portion of example output
related to semaphores:
* * IPC Semaphores * 150 semaphore identifiers (SEMMNI) 500 semaphores in system (SEMMNS) 380 undo structures in system (SEMMNU) 125 max semaphores per id (SEMMSL) 50 max operations per semop call (SEMOPM) 50 max undo entries per process (SEMUME) 32767 semaphore maximum value (SEMVMX) 16384 adjust on exit max value (SEMAEM)(These are just examples, not recommendations.)
The kernel semaphore resources are controlled in the file
/etc/system
with statements like this, which override the
default values.
set semsys:seminfo_semmns=3000 set semsys:seminfo_semmni=100 set semsys:seminfo_semmsl=1200 set semsys:seminfo_semvmx=32767 set semsys:seminfo_semopm=100 set semsys:seminfo_semmnu=500(These are just examples, not recommendations.)
When the settings are modified in /etc/system
, the system
must be rebooted for the change to take effect.
The parameters that are of interest when running Apache/IHS are listed
below, along with requirements for AcceptMutex sysvsem
.
This does not take into account anything other than the accept mutex.
Note that 3rd party modules used with the web server can use semaphores
on their own. "Apache/IHS" refers only to the server and standard
modules, when the directive "AcceptMutex sysvsem" has been chosen.
semsys:seminfo_semmni - SEMMNI - semaphore identifiers
The web server will only use one per web server instance for the accept mutex.
If there are not enough semaphore identifiers available, Apache/IHS 1.3 will write this to error log:
semget: No space left on deviceApache/IHS 2.0 will write this to error log:
[Thu Aug 28 17:26:02 2003] [emerg] (28)No space left on device: Couldn't create accept locksemsys:seminfo_semmns - SEMMNS - semaphores in system
The web server will only use one per web server instance for the accept mutex.
If there are not enough semaphores available, the symtom is the same as shown above under SEMMNI.
semsys:seminfo_semmnu - SEMMNU - undo structures in system
The web server will use one undo structure per child process for the accept mutex, so figure out how many child processes there can be in your configuration to determine how to adjust this.
Apache/IHS 1.3, or Apache 2.0 with prefork MPM: use the value of MaxClients
Apache/IHS 2.0 with worker MPM: if MaxRequestsPerChild=0 and idle child cleanup is disabled, use MaxClients/ThreadsPerChild
otherwise, use MaxClients
If there are insufficient undo structures, the web server could fail at any time (perhaps far after initialization when a new child process is created to handle more load). The message will look like this:
Apache/IHS 1.3
accept_mutex_on: No space left on device [Thu Aug 28 17:37:50 2003] [alert] Child 604 returned a Fatal error... Apache is exiting!Apache/IHS 2.0
[Thu Aug 28 17:44:42 2003] [emerg] (28)No space left on device: apr_proc_mutex_lock failed. Attempting to shutdown process gracefully.Occasionally you may need to adjust the number of web server child processes to handle increased load. That requires a restart of the web server. But that may also require the number of undo structures to be adjusted, which requires a reboot of the machine. Thus, it is wise to overallocate the number of kernel undo structures to allow the number of child processes to be increased without rebooting the machine.
semsys:seminfo_semmsl - SEMMSL - max semaphores per id
One is sufficient for the web server accept mutex.
semsys:seminfo_semopm - SEMOPM - max operations per semop call
One is sufficient for the web server accept mutex.
semsys:seminfo_semume - SEMUME - max undo entries per process
One is sufficient for the web server accept mutex.
Solaris documentation on tunable parameters for SysV semaphores is at http://docs.sun.com/db/doc/806-7009/6jftnqsj6?a=view.
(This section last modified .)