Skip to content

Logging

When an application in a Docker container emits logs, they are sent to the application’s stdout and stderr output streams.

The container’s logging driver can access these streams and send the logs to a file, a log collector running on the host, or a log management service endpoint.

By default, Docker uses a json-file driver, which writes JSON-formatted logs to a container-specific file on the host where the container is running. You can see the path of json-file using the following command:

docker inspect --format='{{.LogPath}}' NAME|ID

Docker Logging Drivers receive container logs and forwards them to remote destinations or files. The default logging driver is “json-file”. It stores container logs in JSON format on local disk. Docker has a plugin architecture for logging drivers, so there are plugins for open source tools and commercial tools available:

  • Journald – storing container logs in the system journal
  • Syslog Driver – supporting UDP, TCP, TLS
  • Fluentd – supporting TCP or Unix socket connections to fluentd
  • Splunk – HTTP/HTTPS forwarding to Splunk server
  • Gelf – UDP log forwarding to Graylog2

For a complete log management solution, additional tools need to be involved:

  • Log parser to structure logs, typically part of log shippers (fluentd, rsyslog, logstash, logagent, …)
  • Log indexing, visualisation and alerting:
    • Elasticsearch and Kibana (Elastic Stack, also known as ELK stack),
    • Splunk,
    • Logentries,
    • Loggly,
    • Sumologic,
    • Graylog OSS / Enterprise
    • and many more …

To ship logs to one of the backends you might need to select a logging driver or logging tool that supports your Log Management solution of choice. If your tool needs Syslog input, you might choose the Syslog driver.

JSON File logging driver

The local logging driver captures output from container’s stdout/stderr and writes them to an internal storage that is optimized for performance and disk use.

By default, the local driver preserves 100MB of log messages per container and uses automatic compression to reduce the size on disk. The 100MB default value is based on a 20M default size for each file and a default count of 5 for the number of such files (to account for log rotation).

The local logging driver uses file-based storage. The file-format and storage mechanism are designed to be exclusively accessed by the Docker daemon, and should not be used by external tools as the implementation may change in future releases.

To use the local driver as the default logging driver, set the log-driver and log-opt keys to appropriate values in the daemon.json file, which is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\daemon.json on Windows Server. For more about configuring Docker using daemon.json, see daemon.json.

The following example sets the log driver to local and sets the max-size option.

{ "log-driver": "local",
  "log-opts": {
  "max-size": "10m" } }
Restart Docker for the changes to take effect for newly created containers. Existing containers do not use the new logging configuration.

You can set the logging driver for a specific container by using the --log-driver flag to docker container create or docker run:

$ docker run --log-driver local --log-opt max-size=10m alpine echo hello world

Options

The local logging driver supports the following logging options:

Option Description Example value
max-size The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (k, m, or g). Defaults to -1 (unlimited). --log-opt max-size=10m
max-file The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. Only effective when max-size is also set. A positive integer. Defaults to 1. --log-opt max-file=3
compress Toggles compression for rotated logs. Default is disabled. --log-opt compress=true

This example starts an alpine container which can have a maximum of 3 log files no larger than 10 megabytes each. 

$ docker run -it --log-driver local --log-opt max-size=10m --log-opt max-file=3 alpine ash

JSON File logging driver

By default, Docker captures the standard output (and standard error) of all your containers, and writes them in files using the JSON format. The JSON format annotates each line with its origin (stdout or stderr) and its timestamp. Each log file contains information about only one container.

{"log":"Log line is here\n","stream":"stdout","time":"2019-01-01T11:11:11.111111111Z"}

To use the json-file driver as the default logging driver, set the log-driver and log-opts keys to appropriate values in the daemon.json file, which is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\ on Windows Server. For more information about configuring Docker using daemon.json, see daemon.json.

The following example sets the log driver to json-file and sets the max-size and max-file options. 

{ "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3" } }
Restart Docker for the changes to take effect for newly created containers. Existing containers do not use the new logging configuration.

You can set the logging driver for a specific container by using the --log-driver flag to docker container create or docker run:

$ docker run --log-driver json-file --log-opt max-size=10m alpine echo hello world

Options

The json-file logging driver supports the following logging options:

Option Description Example value
max-size The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (k, m, or g). Defaults to -1 (unlimited). --log-opt max-size=10m
max-file The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. Only effective when max-size is also set. A positive integer. Defaults to 1. --log-opt max-file=3
labels Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon accepts. Used for advanced log tag options. --log-opt labels=production_status,geo
env Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon accepts. Used for advanced log tag options. --log-opt env=os,customer
env-regex Similar to and compatible with env. A regular expression to match logging-related environment variables. Used for advanced log tag options. --log-opt env-regex=^(os|customer).
compress Toggles compression for rotated logs. Default is disabled. --log-opt compress=true

This example starts an alpine container which can have a maximum of 3 log files no larger than 10 megabytes each. 

$ docker run -it --log-opt max-size=10m --log-opt max-file=3 alpine ash

Fetch the logs

The logs of a container can be fetched by:

docker logs [OPTIONS] CONTAINER

The docker logs command batch-retrieves logs present at the time of execution.

This command is only functional for containers that are started with the json-file or journald logging driver.

Name, shorthand Description
--details Show extra details provided to logs
--follow , -f Follow log output
--since Show logs since timestamp (e.g. 2013-01-02T13:23:37) or relative (e.g. 42m for 42 minutes)
--tail Number of lines to show from the end of the logs
--timestamps , -t Show timestamps
--until Show logs before a timestamp (e.g. 2013-01-02T13:23:37) or relative (e.g. 42m for 42 minutes)

Create a container which write the time to output as follows: 

docker run --name checktime -d busybox sh 
    -c "while true; do $(echo date); sleep 10; done"

  • read the logs of the last 2 minutes
  • read only the last 3 logs
  • read the last 10 logs with their timestamps
  • follow the logs of the container