The logging system within libmtev represents a directed acyclic graph of
input-output chains. Each node in the graph has a unique name and is called
a "log_stream." Log_streams without a
type attribute have output to downstream nodes
("outlets"). Nodes with a
type attribute have additional output
characteristics (like outputting to a file).
Upon startup, the system will establish several built-in log_streams, only one of
which has a type. The "stderr" log_stream has a type of
file and an output
filedescriptor of 2. Other log_stream are setup and and configured to have
the "stderr" log_stream as their outlet. These log_streams are called: "error", "notice",
"debug." The correspond to the global logging symbols in the C API:
For more information on logging via the API, see the development section
of this documentation relaated to logging. The "debug" log_stream is
disabled by default.
Logs are hierarchical in nomenclature as a convenience. If, in your code, you request a log named "error/foo" and no such log exists in the configuration, a new untyped log will be created and its outlet will be set to "error". This is recursive, so "debug/myapp/facility1" will (unless configured otherwise) outlet to "debug/myapp" which will outlet to "debug." This makes it very simple to semantically separate logs into new error and debugging facilities without worrying about them being lost, while providing the flexibility to configure where things go if other outcomes are desired.
All logging configuration exists within the top-level XML node
Individual log_streams are declared using
<log> stanzas and outlets are
<outlet> stanzas. A log_stream uses all
that are its direct child or direct child of any ancestor node.
"1.0" encoding="utf8" standalone="yes" xml version=<application> <logs> <log name="internal" type="memory" path="10000,1000000" require_env="MEMLOG"/> <log name="logfile" type="file" path="/var/log/app.log" rotate_bytes="10000000" retain_bytes="50000000" timestamps="on"/> <log name="http/access" type="jlog" path="/var/log/app-http.feed(*)"/> <console_output> <outlet name="stderr"/> <outlet name="internal"/> <outlet name="logfile"/> <log name="error"/> </console_output> <components> <error> <outlet name="error"/> <log name="error/example"/> <log name="error/sample"/> </error> <debug> <outlet name="debug"/> <log name="debug/example" disabled="true"/> </debug> </components> </logs> </application>
Let's walk through this sample file to understand what's going on. First there
<log> stanzas establishing log_streams named "internal", "logfile",
"http/access", "error", "error/example", "error/sample", and "debug/example".
Starting at the end, the "debug/example" log_stream is declared in a disabled state
disabled="true" attribute. If walk it and its anscestors we find one
<outlet name="debug"/> child. This log_stream has no type, so any messages sent
into this log are only output to its outlet "debug." You'll notice that no
log_stream named "debug" is declared. We rely on the built-in "debug" log which
is setup to output to "stderr". Also, because we did not declare a "debug"
disabled="false", the default state remains disabled.
The "error/example" and "error/sample" log_stream are similarly configured to output to the "error"
log_stream as its outlet. But, we've declared the "error" log_stream in this configuration so that
we can manipulate its outlets. The
nodes have no special meaning by name; they are simply used as descriptive
hierarchical containers to allow us to share outlet configuration and to
logically isolate our intentions.
The "error" log_stream already exists as a built-in log_stream. The declaration here
is used to set outlets to the three log_streams named: "stderr", "internal", and "logfile".
As before, we use an arbitrarily named node to contiain the declaration logically; this
The "internal" log_stream is of type
memory which uses an in-memory ring buffer
to store recent log lines. We have a limit ot 10000 log lines and 1000000 bytes. It
is also only active if the environment variable INMEM is set.
The "logfile" log_stream is of type
file and will auto-rotate files as they hit
10 million bytes and delete old log files as the cumulative space consumed
exceeds 50 million bytes. Timestamps are turned on for this log_stream.
The "http/access" log_stream is of type
jlog which is create a Jlog
journaled log for external consumption.
This optionally requires conditions around an environment variable. See
If "on"/"true", additional debugging information (like thread ID) is injected into logged lines.
If "on"/"true", the name of the log is injected into logged lines.
If "on"/"true", timestamps are injected into logged lines.
If "on"/"true", the stream is disabled and attempts to log to the facility will result in a single branch instruction.
Specifies the number of seconds over which to deduplicated consecutive, identical log lines. The default is 5 seconds.
The memory log_stream type establishes an internal ring buffer in memory. There are APIs (including REST endpoints) to retrieve the contents of this ring buffer. Additionally, if the process crashes one can examine the contents of the ring buffer with a debugger.
pathattribute takes two numbers comma separated. The first number is the maximum number of log lines to be retained. The second number is the maximum number of bytes to be retained. The implementation will not exceed either limit.
The file log_stream type is used to drive writing to ordinary files using the POSIX API. It provides both time-based and size-based retention management capabilities.
The path is the filename to which log data should be written.
Specifies how many seconds of log data should be written into a file before it is moved aside and a new file is started. (used with
retain_secondsand not with
Specified the number of seconds of data to be retained. If all log data in a rotated file are older than this value, the file will be removed. (used with
rotate_secondsand not with
Specifies how many bytes of log data should be written into a file before it is moved aside and a new file is started. (used with
retain_bytesand not with
Specified the number of bytes of data to be retained. If all log data in a in all rotated files exceed this value, the oldest file will be removed. (used with
rotate_bytesand not with
The jlog log_stream type implements an log output to the Jlog multi-file journalled logging format. Jlog is a segmented write-ahead log that is fast and efficient and supports multiple subscribers with independently maintained process checkpoints.
The path is the Jlog directory to be used. It may optionally be ended with a parenthesized subscriber name. If a name (other than "*") is provided, a subscriber of that name will be added to the Jlog on creation.