Streams

When defining a Stream, you should use a command that follows new writes on logs and print them as they arise.

The command should not print older lines. For example, tail -F /var/log/nginx/access.log will print the last 10 lines first, then follow appended lines. This a problem because restarting reaction will result in the same 10 logs potentially printed multiple times.

Examples of good commands:

Plain file

Follow logs of one file

Command:

tail -F -n0 <FILE>

As an array for reaction: (see Why start, stop, stream and action commands are arrays for an explanation)

['tail', '-F', '-n0', '<FILE>']

Follow multiple files as one stream.

tail -q -F -n0 <FILE1> <FILE2>

As an array for reaction:

['tail', '-qFn0', '<FILE1>', '<FILE2>']

⚠️ tail -f and logrotate

When files are rotated, tail -f may stay on the rotated file and miss new inputs.

Using tail -F instead permits to listen on a specific path, even if the actual file under it changes. See tail’s manpage for more details.

SystemD / JournalD

Logs of one systemd unit

journalctl -fn0 -u <UNIT>

Logs of multiple systemd units

journalctl -fn0 -u <UNIT> -u <UNIT>

Docker

Logs of one container

docker logs -fn0 <CONTAINER>

Logs of all the services of a docker compose file

docker compose --project-directory /path/to/directory logs -fn0

Complex commands that need a shell

Multiple commands as one stream

We saw before how to follow multiple files as one stream.

This alternative method can work for any command: sh will launch multiple commands in background, then wait until all of them exit.

sh -c 'tail -F -n0 <FILE1> & tail -F -n0 <FILE2> & wait'

['sh', '-c', 'tail -F -n0 <FILE1> & tail -F -n0 <FILE2> & wait']

Glob (*) patterns

We may also need a shell when we want to use glob (*) patterns.

In that specific case, we only need the shell during startup, but after the glob has been expanded, it is not required during runtime. So we can tell the shell to replace itself by the command to execute, with the standard exec keyword.

sh -c 'exec tail -Fn0 /var/log/nginx/*access.log'

['sh', '-c', 'exec tail -Fn0 /var/log/nginx/*access.log']

Non UTF8 streams

Since reaction v2.1.0, reaction just ignores non UTF8 data. It will be stripped from the log lines.

Non-UTF8 data previously aborted the stream.

stdout/stderr

Since reaction v2, stdout and stderr are both read.

If you run reaction v1, you may use this trick to merge both in stdout for reaction:

⚠️ docker logs print program’s stderr to stderr as well, and reaction only reads stdout before v2.0.0. We might need to capture stdout and stderr if running reaction v1 and your Docker container logs to stderr:

cmd: ['sh', '-c', 'exec docker logs -fn0 <container> 2>&1']

There is virtually no overhead, as the sh process replaces itself with the docker logs command.