How To Rotate Container Logs on The Master and Worker Nodes
Problem
Container logs are utilizing all the disk space on the master and worker nodes.
Environment
- Platform9 Managed Kubernetes - All Versions
- Docker
Cause
The docker service is not configured to rotate the container logs.
Resolution
- Back up the /etc/docker/daemon.json file on the affected node if present. Add the log rotation config on the node in the file /etc/docker/daemon.json file as shown below.
Change the values for max-size and max-file parameter as per your requirement.
$ less /etc/docker/daemon.json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "50m",
"max-file": "5"
}
}
Perform the below steps on one node at a time to avoid downtime.
- Stop the Platform9 Managed Kubernetes stack by executing commands in the below sequence.
$ sudo systemctl stop pf9-hostagent pf9-nodeletd
$ sudo /opt/pf9/nodelet/nodeletd phases stop
- Restart the docker service.
$ systemctl restart docker
- Start the Platform9 Managed Kubernetes stack by executing the commands in the below sequence.
$ sudo /opt/pf9/nodelet/nodeletd phases start
$ sudo systemctl start pf9-hostagent pf9-nodeletd
- Verify the changes took effect on the container by running below command.
$ sudo docker ps | grep -i <conmtainer-name>
$ sudo docker inspect <container-id>| grep -A6 -i LogConfig
"LogConfig": {
"Type": "json-file",
"Config": {
"max-file": "5",
"max-size": "50m"
}
},
Was this page helpful?