Module Name: imfile
Author: Rainer Gerhards <rgerhards@adiscon.com>
Description:
Provides the ability to convert any standard text file into a syslog message. A standard text file is a file consisting of printable characters with lines being delimited by LF.
The file is read line-by-line and any line read is passed to rsyslog's rule engine. The rule engine applies filter conditions and selects which actions needs to be carried out. Empty lines are not processed, as they would result in empty syslog records. They are simply ignored.
As new lines are written they are taken from the file and processed. Please note that this happens based on a polling interval and not immediately. The file monitor support file rotation. To fully work, rsyslogd must run while the file is rotated. Then, any remaining lines from the old file are read and processed and when done with that, the new file is being processed from the beginning. If rsyslogd is stopped during rotation, the new file is read, but any not-yet-reported lines from the previous file can no longer be obtained.
When rsyslogd is stopped while monitoring a text file, it records the last processed location and continues to work from there upon restart. So no data is lost during a restart (except, as noted above, if the file is rotated just in this very moment).
State FilesRsyslog must keep track of which parts of the monitored file are already processed. This is done in so-called “state files”. These files are always created in the rsyslog working directory (configurable via $WorkDirectory).
To avoid problems with duplicate state files, rsyslog automatically generates state file names according to the following scheme:
As a concrete example, consider file /var/log/applog
is
being monitored. The corresponding state file will be named
imfile-state:-var-log-applog
.
Note that it is possible to set a fixed state file name via the deprecated “stateFile” parameter. It is suggested to avoid this, as the user must take care of name clashes. Most importantly, if “stateFile” is set for file monitors with wildcards, the same state file is used for all occurrences of these files. In short, this will usually not work and cause confusion. Upon startup, rsyslog tries to detect these cases and emit warning messages. However, the detection simply checks for the presence of “*” and as such it will not cover more complex cases.
Note that when $WorkDirectory is not set or set to a non-writable location, the state file will not be generated. In those cases, the file content will always be completely re-sent by imfile, because the module does not know that it already processed parts of that file.
MetadataThe imfile module supports message metadata. It supports the following data items
filename
Name of the file where the message originated from. This is most useful when using wildcards inside file monitors, because it then is the only way to know which file the message originated from. The value can be accessed using the %$!metadata!filename% property.
Metadata is only present if enabled. By default it is enabled for input() statements that contain wildcards. For all others, it is disabled by default. It can explicitly be turned on or off via the addMetadata input() parameter, which always overrides the default.
Configuration Directives:
Module Directives
Action Directives
This is a legacy setting that only is supported in polling mode. In inotify mode, it is fixed at 0 and all attempts to configure a different value will be ignored, but will generate an error message.
Please note that future versions of imfile may not support this parameter at all. So it is suggested to not use it.
In polling mode, if set to 0, each file will be fully processed and then processing switches to the next file. If it is set to any other value, a maximum of [number] lines is processed in sequence for each file, and then the file is switched. This provides a kind of mutiplexing the load of multiple files and probably leads to a more natural distribution of events when multiple busy files are monitored. For polling mode, the default is 10240.
Default: on
This parameter controls if state files are deleted if their associated main file is deleted. Usually, this is a good idea, because otherwise problems would occur if a new file with the same name is created. In that case, imfile would pick up reading from the last position in the deleted file, which usually is not what you want.
However, there is one situation where not deleting associated state file makes sense: this is the case if a monitored file is modified with an editor (like vi or gedit). Most editors write out modifications by deleting the old file and creating a new now. If the state file would be deleted in that case, all of the file would be reprocessed, something that’s probably not intended in most case. As a side-note, it is strongly suggested not to modify monitored files with editors. In any case, in such a situation, it makes sense to disable state file deletion. That also applies to similar use cases.
In general, this parameter should only by set if the users knows exactly why this is required.
Default: see intro section on Metadata
This is used to turn on or off the addition of metadata to the message object.
Currently none.
Sample:
The following sample monitors two files. If you need just one,
remove the second one. If you need more, add them according to the
sample ;). This code must be placed in /etc/rsyslog.conf (or wherever
your distro puts rsyslog's config files). Note that only commands
actually needed need to be specified. The second file uses less
commands and uses defaults instead.
Legacy Configuration Directives:
Note: in order to preserve compatibility with previous versions, the LF escaping in multi-line messages is turned off for legacy-configured file monitors (the "escapeLF" input parameter). This can cause serious problems. So it is highly suggested that new deployments use the new input() statement and keep LF escaping turned on.
Sample:
The following sample monitors two files. If you need just one,
remove the second one. If you need more, add them according to the
sample ;). This code must be placed in /etc/rsyslog.conf (or wherever
your distro puts rsyslog's config files). Note that only commands
actually needed need to be specified. The second file uses less
commands and uses defaults instead.
[rsyslog.conf overview] [manual index] [rsyslog site]
This documentation is part of the
rsyslog project.
Copyright © 2008-2014 by Rainer
Gerhards and Adiscon.
Released under the GNU GPL version 3 or higher.