log_files_turing.png

The instrument control software of ARGOS is a distributed system. It is unavoidable that problems will occur. In this case, a logging facility eases diagnosability of problems.

Logging Conventions

The ARGOS ICS provides the following log levels.
Name Description Example
Fatal Computer programs failed to work correctly. They should terminate. Basdard starts with a missing configuration file.
Error These log level messages require operator intervention. The connection to the Laser device breaks.
Warning Operators must expect that the Argos system fails to work properly. Temperature for lasers exceeds some threshold.
Notice These messages contain events that have a impact on the Argos software system state. The laser pulse frequency is set.
Debug For tracking down problems, log messages of this level support the engineer. The laser water temperature is read back from the hardware.

Requirements

  1. Argos ICS uses only the above mentioned log levels.
  2. Each device server must have a facility to change the log level during runtime.

Limitations of the Current Logging System

Argos ICS uses the logging sytem from TwiceAsNice (TaN). At the moment (2012-02-10), this logging system has the following limitations:
  1. Filtering only by log level. TaN lacks filtering by modules or components, e.g. Java logging and Python logging.
  2. No forwarding to central logging place like syslog.

Link To Logging Practices

  1. Log4j Best Practices This article claims that good logs are usable for grep (greppable logs)
  2. Discussion about best logging practices
  3. Blog article about good logging practicies
  4. Log check list
  5. Article "Effective logging practices ease enterprise development"

Links

  1. Filters for rsyslog

Rsyslog Configuration

For CentOS 6, /etc/rsyslog.conf requires these additional lines:

$template ArgosFormat,"%msg:2:4096%\n"
user.*                                                  /var/log/argos.log;ArgosFormat

Note: This assumes that the length of each log message is below 4096!

Clean /var/log/message

All argos specific log messages will go to /var/log/argos.log. The following entry in /etc/rsyslog.conf avoids the log messages of the services to appear in /var/log/message:

*.info;mail.none;authpriv.none;cron.none;user.none                /var/log/messages

Logrotate

Argos logging files can grow up very quickly. A logrotate configuration file is automatically generated from the index file during installation of the common software. In order to trigger the rotation of the ARGOS logs, a cron task must be invoked. Current setup defines a daily rotation, including compression, and 20 days of backup. Empty log files will not be rotated.

In /etc/crontab, the entry depends on the host:

sx-lalas

dx-lalas

  0 7  *   *   *  root /home/argos/bin/argos_dx_lalas_logrotate

Disabling SELinux

Cron fails to execute the logrotate scripts because of SELinux. Here is the way to disabling SELinux in file /etc/selinux/config :

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

After the next reboot, SELinux is disabled.

To disabling SELinux without rebooting, this command will help:
echo 0 >/selinux/enforce
Topic revision: r3 - 01 Aug 2018, AndrewColson
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback