Download from
http://www.nyangau.org/la/la.zip.
tail -f is the friend of anyone doing complicated problem
investigation.
Often there are multiple logs to be watched, and some kind of sequential timed based interleaved composite log is desired. So that you can see that program X complained about something, before program Y did.
The Log Accumulator is designed to make the monitoring and correlation of multiple logs easier.
You tell it to monitor a variety of logs, and it uses poll to
get data from them all.
Completed lines of text get output to the display.
A simple example :-
% la add /tmp/prog-X-log LA: /tmp/prog-X-log added, opened /tmp/prog-X-log: warning, low memory condition add /tmp/prog-Y-log LA: /tmp/prog-Y-log added, opened /tmp/prog-X-log: very low memory condition, closing connections /tmp/prog-Y-log: connection to peer lost ^D %
As the Log Accumulator is running, you can add more logs to be watched
simply by typing more add commands.
This can be seen in the example above.
The Log Accumulators responses are always preceeded by LA:
so that you can easily tell them apart from logged data.
Logged data is always preceeded by the log name.
In the examples, % or # preceeded items are things
typed at the shell prompt, and other text are commands typed into the
Log Accumulator itself.
The log name is the log filename by default. But log filenames can be long and cumbersome, so a shorter name can be used :-
% la add /tmp/prog-X-log X LA: X added, opened add /tmp/prog-Y-log Y fg1 LA: Y added, opened X: restarted Y: connection to peer re-established ^D %
Options may be set when the logged is added, or later
changed using the opt command.
In the example above the Y log has the fg1 option,
which means that its logs get displayed in ANSI colour 1 (red).
Here is how we change it to ANSI colour 2 (green) :-
opt Y fg2 LA: Y options set
Only use the colour options if you're using a suitable terminal, such as
vt100 or similar, ANSI, Xterm etc..
Sometimes when you try to add a log, it won't be possible to open it straight away (perhaps because it doesn't yet exist). You'll see something like :-
add /tmp/log-A A LA: A added, can't open
Later on, you can ask the Log Accumulator to try to open any logs you've added that couldn't be opened by :-
open A LA: A opened
There is an auto open option, called ao that you can
apply to a log when you add it, or later using opt :-
add /tmp/to-be-created 2b ao LA: 2b added, can't open
Later, when the log is created, you might the following :-
LA: 2b auto-opened 2b: To be or not to be, that is the question
Auto-open works by the Log Accumulator continually attemping to open any
logs with the ao option.
This could burn quite a few CPU cycles.
Use sparingly.
Not every program that generates logs puts a timestamp on every line. You can make the Log Accumulator add one like so :-
add /tmp/mylog mylog time mylog: [Sat Jul 26 17:05:31 2003] my event happened
Alternatively the opt command can be used to add this :-
add /tmp/mylog mylog mylog: my event happened opt mylog time mylog: [Sat Jul 26 17:15:31 2003] my next event happened
Timestamps are in local time.
Normally logs operate in 'cooked' mode. ie: the following character sequences are folded as indicated :-
a b \b c -> a c
and the \r character is ignored.
Obviously this feature is designed to accomodate program output whereby
output gets continually updated on the same place on the screen.
eg: A program might display 0% done, do some work, backspace
and display 10% done etc..
This is quite good for program output, but programs such as shells using
getline tend to output various terminal dependant escape
sequences during the command line editing process, and this feature isn't
sophisticated anough to cope with that.
The cooked and nocooked options can be used to
change this behaviour.
When you decide you no longer want to watch a log, you can use :-
del Y LA: Y closed, deleted
Deleting doesn't actually delete the log file. It just deletes it from the Log Accumulators list of things to watch.
The following example shows a listing of the logs, an attempt to open any which are closed, and finally to close them all.
show * LA: /tmp/x (x) is open LA: /tmp/y (y) is not open open * LA: y can't be opened close * LA: x closed
Notice that we use * as a wildcard to mean all logs.
We can do this most places, just not on the add command.
The Log Accumulator works a little like tail -f.
When the Log Accumulator opens a log file, it immediately seeks to its end. So it only reads and displays events which are appended to the log from that point in time onwards.
Also, if a log is truncated/recreated, the Log Accumulator will not explicitly notice this. Consider the following sequence :-
Making the Log Accumulator close and reopen a log causes it to forget its 'position' into the log.
Beware programs which do automatic log rotation.
Some programs display a row of dots (or other partial lines) as they do their job. Log Accumulator only displays lines when they are completed. Sometimes it is useful to be able to take a peek of uncompleted lines.
install.log: Decompressing data, could take a while system.log: Disk space low (10% free) peek * LA: peek install.log: Decompressed ............ system.log: Disk space low (5% free) install.log: Decompressed ........................... Done. install.log: Installation complete.
Very often you'll set up the Log Accumulator the same way. Just put the commands in a file, and feed it in when you start. More than one file can be passed.
% cat setup.la add /tmp/server-X-log X fg4 add /tmp/server-Y-log Y fg6 ao tee /tmp/combined.log % la setup.la LA: X added, opened LA: Y added, can't open LA: tee-ing to /tmp/combined.log X: low memory threshold reached
The help command shows the full command set :-
help
LA: Commands :-
LA: add <filename> [<name> {<option>}]
LA: opt <name> {<option>}
LA: del <name>
LA: show <name>
LA: peek <name>
LA: open <name>
LA: close <name>
LA: echo <text>
LA: cat <filename>
LA: tee [<filename>]
LA: exit
LA: help
LA: Where :-
LA: <filename> is a file, pipe or device
LA: <name> is a shorter name to refer to the log by (or *)
LA: <option> is [no]ao, [no]time, [no]cooked, fg<col>, nofg, bg<col>, nobg
LA: <col> is 0-7 for black,red,green,yellow,blue,magenta,cyan,white
LA: <text> is any text
From here we can see that we can also cause our own text to be entered into
the output stream (using echo).
We can also put the contents of a file into the output stream using
cat.
The output stream can be sent to a file as well as the screen. Note that if you do this, the ANSI colour escape sequences are not sent to the file, they only go to the stdout output stream (ie: normally the terminal).
Putting it all together in an example :-
tee /tmp/test-log LA: tee-ing to test-log cat /tmp/pidfile /tmp/pidfile: 12345 echo About to kill server LA: echo About to kill server server-log: SIGTERM received, aborting cat /tmp/pidfile LA: can't open /tmp/pidfile tee LA: tee-ing ended
Some programs naturally generate logs, or can be made to.
Logs of shell interactions can be made with the UNIX script
command, which creates a nested shell that is logged - eg:
# script -a -f /tmp/root-shell Script started, file is /tmp/root-shell # install.sh Product successfully installed # exit Script done, file is /tmp/root-shell #
Afterwards :-
% cat /tmp/root-shell Script started on Sat Jul 26 13:34:00 2003 # install.sh Product successfully installed # exit Script done on Sat Jul 26 14:14:26 2003 %
The -f argument to script is present on the
Linux version, but not necessarily on others.
It causes 'flushing after each write', ie: data is written to the log file
as and when it is displayed.
Without such an option, shell output and resulting log output do not
interleave correctly in the Log Accumulator output, as the shell output gets
buffered until there is a lot of it.
CA eTrust Access Control was formerly Memco SeOS, and this is reflected in the names of the files comprising the product.
This is a security product, which records an audit log of attempted violations. Unfortunately, by default, this isn't written to an ever growing text file.
It includes a program called seaudit which can be made to
display the violations in a given time interval, but once done it terminates.
So it and doesn't behave like the -f in the
tail -f command, and so isn't useful.
Fortunately to support centralised logging, eTrust Access Control includes
a log emitter (called selogrd) and a collector (to be run on a
central host).
The trick is to ensure that /usr/seos/log/selogrd.cfg
contains :-
Rule#1 file /tmp/audit.log .
What is logged can be selected via include and
exclude primitives, along with other 'places' to emit the
audit violations to.
See the selogrd manual page for details.
selogrd checks for new audit violations every few seconds,
as specified in /usr/seos/seos.ini in section
[selogrd] under setting Interval.
Its normally 5, but might be better with a smaller value. When using Log Accumulator to merge logs from various commands, and from eTrust, this would ensure the security violation was display closer to the command which caused it.
Note: Despite there appearing to be no specific/visible rule covering
/usr/seos/seos.ini, you can't amend this file unless
eTrust is disabled.
You then ensure that selogrd is running, either by
modifying /etc/inittab, or just simply running it.
You can then add the output file to the Log Accumulator.
Use a wide screen and window!
Use a filtering tool to process the resulting composite logs.
Log Accumulator is only a very small and simple program. But it does have the potential to significantly improve the recording of actions taken and results seen in most formal testing scenarios.