Use logrotate for log rotation and automatic archiving.

logrotate can rotate, compress, and delete old log files. It is installed by default in most Linux distributions and automatically runs every day.

logrotate configuration file#

Create and edit the file /etc/logrotate.d/app. Generally, the configuration file /etc/logrotate.conf will have a configuration like include /etc/logrotate.d, which automatically loads the configuration files in the /etc/logrotate.d directory.

/root/app/log/app.log {
    daily          # rotate daily
    missingok      # do not report error if log file is missing
    rotate 100     # keep 100 log files
    compress       # compress old log files
    delaycompress  # delay compression until next rotation
    notifempty     # only rotate if log file is not empty
    dateext             # use current date as log file extension
    dateformat -%Y%m%d  # set date format
    olddir /root/app/log/old # set directory to store old log files (must be created beforehand)
    sharedscripts # run postrotate script once after all logs are rotated, instead of running it for each file
    postrotate    # script to run after log rotation
        /usr/bin/docker restart app
    endscript     # end of script
}

Note: Restarting the service may cause a network disconnect, so this needs to be taken into account. You can use the following method to test if the network will be disconnected during the restart process:

for i in {1..1000}; do { curl -I https://example.com/; sleep 0.3; } ; done

Testing#

You can directly use the logrotate command to test. Use the -d option for debug mode, -f for force execution, and -v for verbose mode to see the execution process. In debug mode, the command will only print the execution process without actually executing it, which can be used for testing.

logrotate -dfv /etc/logrotate.d/app

After disabling debug mode, the result will be as follows:

$ tree
.
├── app.log
└── old
    ├── app.log-20231116.gz
    └── app.log-old

1 directory, 3 files

Possible Issues#

Permissions: I am currently using the root user, so I did not encounter any permission issues. If you are using another user, you may encounter permission issues and need to be aware of them.
Restart: The postrotate script is executed after log rotation, so if you are restarting the service, it may cause a network disconnect. This needs to be taken into account.
Rotation based on file size: If rotation is based on file size, it may result in log files being truncated. This needs to be taken into account. I did not encounter this issue as I am not using rotation based on file size.

References#

Linux Log Rotation Tool logrotate: Introduction and Configuration - A well-written article that explains the principles and configuration of logrotate.
logrotate(8) - Linux man page - Official documentation for logrotate, providing detailed configuration information.