rotatelogs
write to log files over 2GB?Platform | 1.3 releases? | 6.1 and earlier? | 7.0 and later? |
---|---|---|---|
AIX, HP-UX/PA-RISC, Linux, Solaris/SPARC | No | No | Yes |
HP-UX/PA-RISC, Solaris/x86_64, z/OS | N/A | Yes | Yes |
Windows | No | Yes | Yes |
This applies to IBM HTTP Server 2.0 through 6.1.
The web server and rotatelogs use file access interfaces provided by the Apache Portable Runtime (APR) library bundled with IBM HTTP Server. The APR provided with IBM HTTP Server on these platforms cannot support file offsets larger than 2GB without introducing an API incompatibility, which would break all current plug-in modules written for IBM HTTP Server. However, it was possible to enable large file support for applications which only append to files without introducing an API incompatibility.
The web server's internal support for error and access log files only appends to the end of log files and does not use file offsets, so the modifications to APR allow large log files when using the internal web server support.
The rotatelogs application interacts with log files in a more complex manner, including the use of file offsets. Thus, the APR changes don't enable large file support in rotatelogs.
For releases 7.0 and later, the bundled APR allows rotatelogs to use large file offsets. Note that since rotatelogs has a static copy of the APR library, rotatelogs from later releases can be used as a piped logger in previous releases.
No, the rotate operation will not occur until IHS logs another request. If your configuration specifies that rotatelogs performs the rotation operation after 86400 seconds, and if IHS receives no requests after 86400 seconds have elapsed, the new log file will not yet be created. Then, when rotatelogs receives its next request to log, it will create the new log file and close the old one.
No. However, data may be buffered in the operating system kernel after the web server writes the data but before rotatelogs can read it. This time is usually very brief.
Other programs can be used to filter data seen by rotatelogs, and those programs may introduce buffering. Example:
CustomLog "|grep -v \b200\b | /opt/IHS/bin/rotatelogs /opt/IHS/logs/grepped 86400" common
In this case, grep will buffer data internally until it has a full buffer, then rotatelogs will see many log records at the same time. (4096 bytes is a typical size for the buffer used by grep.)
Note: IBM HTTP Server 6.0 prior to 6.0.2.1, or IBM HTTP Server 2.0 prior to PK07831, does not support the type of CustomLog directive in the example above.
Example:
CustomLog "|/usr/HTTPServer/bin/rotatelogs -l /www/logs/access_%Y-%m-%d 86400" custom
This example includes the year, month, and day in the log file
name. An example generated filename is
/www/logs/access_2007-11-26
.
For example, if you configure rotatelogs to rotate every 7 days (604800 seconds), a new logfile created January 1-6 1970 would have a timestamp of January 1 1970.
This algorithm can easily be investigated on a shell command line:
date -u --date=@$(($((`date +%s` / 604800)) * 604800))
Apache, especially on Windows, launches rotatelogs a number of times for each time it's used in the configuration, as the configuration is first checked, re-ran, then possibly run again in the child proces (Windows only).
When you rotate based on size, and include seconds in your filename format, you will see multiple files with the second field incrementing.
We suggest not using seconds in general, but especially when rotating based on size.
error writing to logfile xxx messages lost
This message can also occur when there is insufficient free space on the filesystem used for logging.
If this error message occurs, the log file being written to by rotatelogs will be truncated and then new entries will continue to be written to the log.
If you are using Windows and are trying to use large numbers of rotatelogs instances (for example, within a large number of Virtual Hosts), then you may experience errors or rotatelogs processes frequently crashing and being restarted. This is caused by exhaustion of a resource called the "Desktop Heap". For more information on the desktop heap, refer to this article.
Increasing the desktop heap size involves changing registry values. Back up the Windows Registry before making any changes and be very careful when changing any values.
To increase the desktop heap, use the following procedure:
Note that increasing the last value increases the desktop heap size for all services. Do not make the value larger than necessary; each service in the system will consume more resources. While experimenting, we found that a value of 2048 was enough to run 200+ instances of rotatelogs.
rotatelogs is a piped logger which is responsible for switching to a new log file when certain criteria are met. You can choose to use piped logger software from a third party.
Refer to How can I rotate (switch) log files? for additional information on other possible alternatives.
This behavior is the same in versions 6.x, 7.0, and the newer 8.x versions of IBM HTTP Server and is a limitation of using piped loggers such as rotatelogs.
If you attempt to remove a log that still has active references, the delete will not take place and if
you are using a process monitor type tool, you may see an indication such as "DELETE PENDING"
.
The behavior may vary depending upon whether you attempt the delete from the command-line versus doing it from
Windows Explorer, with the latter apparently less likely to have issues.
If mod_mpmstats
is not regularly writing messages from within the
parent process, the parent will retain a reference to the one original error log that is opened when the web
server starts. In this scenario without mod_mpmstats
, the global error log opened
at startup will not be deletable.
For IHS 7.0 and later, mod_mpmstats is included in the product and enabled by default.
For earlier releases, you can obtain the mod_mpmstats
module by downloading the
ihsdiag
package at this url:
Debugging IBM HTTP Server crashes, hangs, high CPU, startup failures
Rotation is usually configured because logs are growing at a rate proportional to the number of requests, which implies the child is actively writing to the erorr log. But in a synthetic test, you might find that no entries are written and the childs lock remains even after the parent has rolled over.
If the original error log still can't be deleted, then it is probable that the child process is retaining a reference to the file. This should happen only if there has been nothing to cause the child to write to the error log, and hence to cause it to move to the new error log. In such a case, the following techniques might help:
MaxRequestsPerChild
' to a non-zero
value. When a new child process gets created, it should not have a reference to the original error log.
This technique still allows windows of time during which the original log cannot be deleted, but if there
is sufficient delay between the rotate and the attempted deletion, this may work in some scenarios.
You must use the full filesystem path to the piped logger. The piped logger
cannot be specified relative to the ServerRoot
. The operative part of the error message in this
FAQ is that the path is not absolute. For rotatelogs, the logfile path must also be absolute (although other piped loggers
might allow you to use a relative path).
It is possible that a more sophisticated, non "rotatelogs" piped logger could coordinate its writes across processes.
It is possible that a more sophisticated, non "rotatelogs" piped logger could arrange to close the file earlier.