The previous article explained how to monitor an entire container in Docker. In addition to monitoring, you should also limit the resource usage of each container so that server performance remains in good condition.

Problem

How to limit resources in Docker?

Solution

If you run the docker stats command, the command will display the resource container as shown in the image below:

As you can see in the image above, there are 2 containers running, and both containers have a memory limit of 941 MB, where the memory size is the size of the memory of the server. This is dangerous because the application in the container can use all the memory or CPU on the server, causing the server to run abnormally. Therefore, you should limit the use of resources in the container.

A. RAM limitation

There are 2 types of memory limitations in Docker:

• The hard limit is a maximum value that cannot be exceeded. When a container exceeds a hard memory limit, Docker takes aggressive actions such as terminating the container so that there will be an OOM or Out Of Memory error in the container. The options used in Docker are ‐‐memory or -m.
• The soft limit is a limit that can be temporarily exceeded. When a soft limit is reached, Docker warns the user but does not take immediate action. The option used is ‐‐memory-reservation.

If you use swap on the server, you can use the ‐‐memory-swap option to allocate available swap memory to the container. This swap memory must be larger than the hard limit, usually twice larger than the hard limit. If you want to use a percentage for memory swap, use the ‐‐memory-swappiness option.

To limit the memory on the new container, use the format below:

docker run -d --memory='hard_limit_value' --name docker_name docker-image

For example, if you want to limit the memory on a container of 512 MB with a soft limit of 256 MB, then I run the command below:

docker run -d --memory='512m' --name nginx nginx

You can immediately change the container memory when the container is running using the format below:

docker update docker_name --memory='hard_limit_value'

For example, I want to change the Redis container memory from 971 MB to 256 MB using the command below:

But when running the command, there is an error as below:

Error Response from Daemon: Cannot Update Container 94DD458F891C08D8E0A15EB00878FC83E5C6660D6AF8Beb0815A3A9040C57F7: Memory Limit Should Be Smaller Than Already Set Time

To overcome the error, update the memory container swap, whose value must be greater than the memory value. Therefore, use the command below to increase the memory of a container:

docker update redis --memory='256m' --memory-swap='512mb'

And the container memory value should have changed as shown below:

B. CPU limitation

Before you limit the CPU in the container, you need to know how many CPU cores there are on your server by using the command below:

nproc

To limit the CPU used in the container, use the ‐‐cpus option to determine how many CPU cores can be used in a container. If your server has two CPUs and you set ‐‐cpus=’1.5′, the container is guaranteed at most one and a half of the CPUs and this is the equivalent of setting ‐‐cpu-period=’100000′ and ‐‐cpu-quota=’150000′ (cpu-period is used with cpu-quota to configure the CPU scheduler with defaults to 100000 microseconds and cpu-quota is used with cpu-period to configure the CPU scheduler). Use the format below to limit the CPU in the container:

docker run -d --cpus='cpu_value' --name docker_name docker_image:tag

You can see the details of the CPU used by a container by using the format below:

docker inspect nginx | grep -e Cpu -e cpu

For example, I want to limit the CPU to 1, so I run the command below:

docker run -d --cpus='1' --name nginx nginx

You can use the ‐‐cpu-shares option for a container to control the share of CPU cycles available, whose default value is 1024. This option is like a soft limit option on memory, so if you run the command below:

docker run -d --cpu-shares='2048' --name webapp1 nginx

If there is more than 1 container on a host and CPU cycles are constrained, then the webapp1 container will receive 2x more CPU than the other containers. You can update the CPU in a running container using the format:

docker update docker_name --cpus='cpu_value'

For example, I have 2 CPUs on the server, and initially, I use all the CPUs in the webapp1 container. Then, I wanted to update the CPU on the container to 1.5, so I ran the command below:

docker update webapp1 --cpus='1.5' nginx

C. HDD limitation

As of this writing (April 2025), the HDD limitation can only be used for btrfs, overlay2, windowsfilter, and zfs storage drivers. For the overlay2 storage driver, the size option is only available if the backing filesystem is xfs and mounted with the pquota mount option. Type the command below to see the Docker settings on the server:

docker info | grep -e 'Filesystem' -e 'Support' -e 'Storage'

Run the command above, and here is the result:

To limit the hard disk in a container, follow the format below:

docker run -d --name nginx --storage-opt size=1g nginx

If I want to limit my hard disk in a container, I run the command below:

docker run -d --name nginx --storage-opt size=1g nginx

But, I have an error like the image below:

docker: Error response from daemon: –storage-opt is supported only for overlay pver xfs with ‘pquota’ mount option

From the image above,  I can’t limit the HDD to the container because my Backing Filesystem in my server still uses extfs, not xfs. So, if I want to limit my HDD in my containers, I have to change my Backing Filesystem in my server. So I made an experiment by turning off Docker and deleting the Docker folder using the command below:

sudo systemctl stop docker
sudo rm -rf /var/lib/docker && sudo mkdir /var/lib/docker

Then I inserted an additional hard disk into the existing server, and then I converted the file system to xfs using the commands below:

sudo mkfs.xfs /dev/sdb1
sudo mount /dev/sdb1 /var/lib/docker

And I added to the /etc/fstab file the script below:

echo '/dev/sdb1 /var/lib/docker xfs defaults,quota,prjquota,pquota,gquota 0 0' | sudo tee -a /etc/fstab

I rebooted the server to test whether the fstab file settings were correct. After that, I tried to create a container by limiting the container’s hard disk to 1GB using the command below:

docker run -d --name nginx --storage-opt size=1g nginx

Note

You can combine more than one restriction above by using one command. For example, if you want to limit memory, CPU, and HDD in a container, then you can run the command below:

docker run -d --name webapp1 -m 512m --cpus=1.5 --storage-opt size=1g nginx

As far as I know, Sysadmin only limits the use of RAM and CPU in Docker, but rarely limits HDD in Docker. If you want to limit resources in Docker, you must discuss it with the developers so that there are no problems with the application in the future.

References

After you run containers in Docker on your server or your Docker Host, you should monitor all existing containers to find out the performance of each container.

Problem

How to monitor containers in Docker?

Solution

There are 2 methods for container monitors in Docker:

A. Via CLI

In CLI, to monitor all containers in Docker, you can use the command:

docker stats

You will see the display as below:

From the image above, you can see that the command displays the results in streaming, and to exit from the command above, press Ctrl+Z or Ctrl-C. If you don’t want to display the results in streaming mode, then use the command below:

docker stats --no-stream 

B. Via Website

If you want to monitor Docker via a website, you can use the Portainer tool. Portainer is a tool for managing containers through a browser that can support Docker host, Docker Swarm, Nomad, and Kubernetes. It has 2 components, namely Portainer Server, which is used to manage containers, networks, and environments, and Portainer Agent is the component installed on another Docker system to enable communication with the server. Portainer has 2 editions, namely Portainer Business Edition or PBE and Portainer Community Edition or PCE, where both editions at the time of this writing (April 2025) have version 2.27.3. This article will discuss how to install Portainer Community Edition. Here are the steps:

1. Create Docker Volume

Type the command below to create a new volume in Docker:

docker volume create portainer_data

2. Install Portainer

Type the command below to install the latest version of Portainer:

docker run -d \
-p 8000:8000 \
-p 9443:9443 \
--name portainer \
--restart=always \
-v /var/run/docker.sock:/var/run/docker.sock \
-v portainer_data:/data portainer/portainer-ce

3. Check the Portainer

The following command can be used to determine whether Portainer is operating or not:

docker ps

4. Access the Portainer

After that, open your browser and type:

https://your_IP_server:9443

There will be an image like below:

A picture similar to the one below will appear when you click the Advanced button:

Click the unsafe link in your browser, and then there will be an image like below:

If you have an error like the picture above, then restart Portainer by running the command below:

docker restart portainer

Enter the desired name and password, then click the Create user button, and you  will see an image below:

The Portainer dashboard will appear. Click the Get Started box like in the above image, and there will be an image below:

There will be an image below:

Click on the red box, and there will be an image below:

I have 4 containers in my server, but I want to detail each container, so I click in the red box, and there will be an image below:

If I want to display the resource of the Redis instance, click the icon in the red box, and there will be a display below:

If I want to access a container, I click the icon like in the red box:

There will be a display like in the image below:

Select the command used in the container and select the desired user. After that, click the Connect button, and there will image like in the image below:

You can access the inside of the container and give the Linux command from your browser to the container. From this tool, you can see the images in your Docker when you click Images, like in the image below:

You can display the Volume in Docker after you click the Volumes, like in the image below:

Note

If you want to monitor Docker on another server using Portainer, you have to install the agent using the command below:

curl -L https://downloads.portainer.io/agent-stack.yml -o agent-stack.yml && docker stack deploy --compose-file=agent-stack.yml portainer-agent

References

youtube.dimas-maryanto.com
docs.portainer.io
phoenixnap.com
musaamin.web.id
letscloud.io

By default, if you want to create a Docker container on your server, you can download the required image directly using the docker pull command. But sometimes, there are some cases where you cannot download the image directly from the internet, and you have to back up the existing image and then restore the image to a server.

Problem

How to back up and restore Docker image(s)?

Solution

I have a server that, due to security issues, cannot be connected to the internet, while on the server, many applications run using Docker. Because it cannot directly download the Docker image, I have to download the Docker image on a server that is connected to the internet, and then the image will be installed on this server.

A. Backup Docker image

To back up a Docker image, use the format below:

docker image save -o image_name.tar image_name:tag

For example, if you want to back up the nginx image, then use the command below:

docker image save -o nginx.tar nginx

Wait until the process is complete, and if it is finished, the backup image file will be formed as shown below:

But use the format below if you want to back up more than one image:

docker image save -o image1_name.tar image2_name:tag ...

I have more than one Docker image on my server, so I want to back up all the images so I can run the images on my other server, which is not connected to the internet. Then use the command below to back up the docker image of more than one Docker image:

docker image save -o all_images.tar alpine redis nginx

And the backup image file should be formed according to the image below:

B. Restore Image Docker

After you back up the Docker image, move your Docker image to the desired server. To restore the Docker image, use the format below:

docker image load -i image_name.tar

So I restored the Docker image backup file using the command below:

docker image load -i nginx.tar

Then the Nginx image will be restored on the server as shown below:

With the same command, you can also restore more than one image using the command below:

docker image load -i all_images.tar

Then all Docker images will be restored on that server, like in the image below:

Note

You can use the command below to back up the Docker image using the format below:

docker save image_name | gzip -c > image_name.tgz

So if you want to back up the Nginx image, use the command below:

docker save nginx | gzip -c > nginx.tgz

The advantage of using this command is that the size of the backup file is much smaller than using the previous command, as shown in the image below:

To restore, use the format below:

gunzip -c filename.tgz | docker load

So if you want to restore more than one image, use the command below:

gunzip -c all_images.tgz | docker load

And the Docker image will be restored on the server.

References

youtube.dimas-maryanto.com
youtube.com
docs.docker.com
stackoverflow.com

A large application that consumes a lot of CPU on a Linux server will cause unusual server conditions. Therefore, you have to limit the use of the CPU for the application.

Problem

How to limit the use of CPU in Linux?

Solution

You can use the cpulimit tool to limit CPU use. Below is the command to install the tool in Linux:

Debian/Ubuntu

sudo apt update
sudo apt-get install cpulimit -y

RockyLinux/AlmaLinux/CentOS

yum install epel-release -y
yum install cpulimit -y

OpenSUSE15

zypper install -y cpulimit

Here are some methods using the cpulimit command:

A. Using ‐-pid option

To use the ‐-pid or -p option, you need to know the PID of an application that you want to limit its CPU usage. To run this cpulimit tool, use the format below:

cpulimit --pid xxx --limit xxx --background

Where PID is the ID number of the ongoing application, you can see by using the format below:

ps aux | grep application_name

The limit is a percentage figure from the CPU that you want to limit for the use of the application. We will use the bash script to simulate this CPU server to be high. Create a high_cpu.sh file and copy the script below

#!/bin/bash

# Simple infinite loop that uses 100% CPU
while true
do
    :  # No-op (no operation) to keep CPU busy
done

Run the command below to give permission and run this bash script:

chmod +x high_cpu.sh
./high_cpu.sh &

Run the top command on another terminal, and as you can see in the image below, the script uses 99 percent of the CPU on the server:

Now run the cpulimit command by limiting this script to run using only 50 percent of the CPU on this server:

cpulimit --pid 1198 --limit 50 --background

If you look at the top command, this script is no longer utilizing 99 percent of the CPU on this server.

The disadvantage of using the ‐-pid option is that if the application is restarted, the PID of the application will change, and you have to change the cpulimit command. So, there is another option where you just write the name of the application in the cpulimit command using the ‐-exe option.

B. Using ‐-exe option

To use the ‐-exe or -e option, you need to know the application name that you want to limit its CPU usage. To run this cpulimit tool, use the format below:

cpulimit --exe application_name --limit xxx --background

We will use the bash script to simulate a CPU server to be high with many PIDs. Create a pids_high_cpu.sh file in the folder /etc and copy the script below:

#!/bin/bash

# Number of processes to spawn
num_processes=5

# Function to keep the CPU busy in each child process
cpu_intensive_task() {
    while true
    do
        :  # No-op command to keep the CPU busy
    done
}

# Spawn the specified number of processes, redirecting output to /dev/null
for ((i=0; i<num_processes; i++))
do
    cpu_intensive_task > /dev/null 2>&1 &  # Run the task in the background and suppress output
done

# Optionally, wait for all background processes to complete (won't happen unless manually killed)
wait

Run the command below to give permission and run this bash script:

chmod +x /etc/pids_high_cpu.sh
/etc/pids_high_cpu.sh &

Run the top command on another terminal. As you can see in the image below, the script has many PIDs and uses a lot of CPU resources on the server:

Run the cpulimit command  to restrict this script to utilizing just 50% of the server’s CPU:

sudo cpulimit --exe pids_high_cpu.sh --limit 50 --background

C. Using ‐-path option

Besides using the ‐-exe option, you can also use the ‐-path option or -P option. To use this option, you have to know the path of the application that you want to limit its CPU usage. Use the format below to run the ‐-path option in the cpulimit command:

cpulimit --path /folder/path/of/the/application --limit xxx --background

For the same case, type the command below to use the cpulimit command with the ‐-path option to limit CPU usage to 65 percent for pids_high_cpu.sh application in the /etc folder:

cpulimit --path /etc/pids_high_cpu.sh --limit 65 --background

Note

If you see an application running that generates a lot of PIDs, as shown in the image below:

From the picture above, you can see that the application Vivaldi has many PIDs. So, you have to write a script to restrict apps with numerous PIDs. Here is a copy of the bash script:

#!/bin/bash

LIMIT=50
PROCESS_NAME="vivaldi-bin"

# Kill existing cpulimit instances related to vivaldi-bin
pkill -f "cpulimit -p" 2>/dev/null

# Get all vivaldi PIDs
PIDS=$(pidof $PROCESS_NAME)

if [[ -z "$PIDS" ]]; then
  echo "[$(date)] $PROCESS_NAME not running."
  exit 0
fi

echo "[$(date)] Limiting CPU for PIDs: $PIDS"

# Start cpulimit for each PID in background
for pid in $PIDS; do
  cpulimit -p "$pid" -l "$LIMIT" > /dev/null 2>&1 &
done

Change the LIMIT and PROCESS_NAME sections according to your needs and permit the script to be executed. If the script runs, it will limit the CPU to the PIDs.

Enter the script into the crontab using the script below: 

#Limit CPU vivaldi
*/30 * * * *     /root/limit_vivaldi.sh
@reboot         /root/limit_vivaldi.sh

The script will restart every 30 minutes, and if the device restarts, the script will start automatically.

References

tecmint.com
linuxsec.org
id.ubunlog.com
linuxsec.org
youtube.com

If you have a virtual machine at GCP, by default, the Linux system will only make one partition / only. If the partition is smaller, then you have to increase the hard disk server size

Problem

How to increase HDD capacity on a VM in GCP?

Solution

Currently, I have a VM Ubuntu Server 24.04 in GCP with an HDD capacity of 10 GB as in the image below:

You can see from the image above that my partition / is very small, and here is the block device in my VM:

I want to increase the HDD capacity to 20 GB without rebooting the server. These are the steps below (recommended as a root user to do the steps below):

1. Edit in the Disks section

You can use the command below to increase the VM’s hard disk to 20 GB in the cloud shell or from your laptop if you have already installed gcloud (change the VM name, size, and zone from the command below):

gcloud compute disks resize vm-cloud --size 20 --zone=us-central1-c

You can also increase the hard disk in GCP by entering GCP, selecting Compute Engine – Disks, and then selecting VM.

After that, click Edit as below:

2. Increase HDD capacity

After that, increase the HDD capacity in the section as shown in the image below:

Change it to 20 GB, and after that, press the Save button so there will be a display like below:

3. Check the block devices

Enter the VM, then we check the block devices using the command:

lsblk

You can see in the picture above that the HDD capacity is 20 GB.

4. Check the partition tables

Then check the partition tables using the command:

parted -l

If you have a warning like in the image above, you can choose Fix or Ignore, but I chose Fix. From the picture above, you can see that the HDD in this VM has number 1 in the /dev/sda partition using the ext4 extension.

5. Resize the partition

 Use the command below to resize the partition:

parted /dev/sda

Then type the command:

resizepart

Enter number 1 (check your partition number in step 5), type Y, and type 100%. After that, type quit to exit the prompt.

6. Read the new partition table

Use the command below to read the new partition table:

partprobe /dev/sda

7. Extend the file system

Use the command below to see the types of filesystems used in your VM:

df -T

To extend the file system, use the command below if you are using ext4 (and I am using this filesystem):

sudo resize2fs /dev/sda1

If you use the xfs filesystem, use the command:

sudo xfs_growfs -d /

But if you use btrfs, then use the command:

sudo btrfs filesystem resize max /

8. Check HDD capacity

Use the df -h command to check the hard disk capacity, and it should match the additional HDD in the GCP (in my case, the HDD capacity is 20 GB):

Note

You should back up the important data on the VM first before following the steps above. However, you can increase the HDD capacity in a VM without doing the steps above by rebooting the VM after changing the HDD capacity in the GCP console (step 2).

References

cloud.google.com
man7.org
medium.com
gist.github.com
youtube.com

The previous article explained how to create a virtual machine in GCP using a GUI template. This article will explain how to create a virtual machine on GCP using the CLI.

Problem

How to create a virtual machine using CLI in GCP?

Solution

These are the steps to create a virtual machine using CLI:

A. Access to the gcloud

If you have already installed gcloud on your laptop, you can go to the next step. But, if you want to use gcloud in your GCP, go to the GCP dashboard, then click the small box as shown in the image below to activate the cloud shell (or you push the G then S button):

There will be a display below at the bottom:

After you click Continue and wait a minute, the screen shown in the picture below will appear:

Click Authorize, and the cloud shell is ready to use. If you want the cloud shell to have a larger screen, you can click the button below:

And the existing cloud shell will be inactive, as shown in the image below, so that the cloud shell will move to a new tab:

B. Run the command

By default, you can use the command below to display the options to create a virtual machine:

gcloud compute instances create --help

From the image above, you can see that you have many options. But actually, you can only use 3 options to make a VM: the zone, machine types, and image options. You have to know that by default, a VM will automatically get a hard drive size of 10 GB, so you don’t need to determine the size of a hard drive on a VM. To see the available zone options, use the command below:

gcloud compute zones list

Use the following command to view the machine type you wish to use:

gcloud compute machine-types list 

To see the available images, use the command below:

gcloud compute images list

So, if you want to create a virtual machine in zone us-central1-c, use machine-type e2-standard-2, and use OS Ubuntu 24.04, use the command below:

gcloud compute instances create vm-cloud \
--zone=us-central1-c \
--machine-type "e2-standard-2" \
--image-project "ubuntu-os-cloud" \
--image-family "ubuntu-2404-lts-amd64" \
--subnet "default"

After that, check the existing VM in GCP using the command below:

gcloud compute instances list

The VM should be made as shown below:

Or you can see the list of the VMs in the VM instances page in the image below:

Note

If you want to create  a VM with a 50GB hard drive, use the command below:

gcloud compute instances create vm-cloud \
--zone=us-central1-c \
--machine-type "e2-standard-2" \
--image-project "ubuntu-os-cloud" \
--image-family "ubuntu-2404-lts-amd64" \
--boot-disk-size=50GB \
--boot-disk-type=pd-standard \
--subnet "default"

References

cloud.google.com
medium.com
diana-moraa.medium.com
youtube.com

I have a Linux Ubuntu server version 22.04, and I want to upgrade to the latest version of Ubuntu.

Problem

How to upgrade Ubuntu to the latest version?

Solution

Before you upgrade your Ubuntu version, I think you have to back up your important data to other devices, and have internet to download the packages needed to upgrade. After that, open port 1022 on your laptop or server if you use the firewall using the below commands:

sudo ufw allow 1022/tcp
sudo ufw reload
sudo ufw status

You should know that the Ubuntu version upgrade process can only be done to one major LTS version. So if you have Ubuntu version 20.04 and want to upgrade to the latest version (version 24.04 in November 2024), you have to do a 2x upgrade process, upgrading to version 22.04 first and then to version 24.04. I have Ubuntu version 22.04, like in the image below:

So, I type the command below:

sudo apt update
sudo apt upgrade -y

After that, reboot the server using the command below:

sudo reboot

After reboot, run the command below:

sudo do-release-upgrade

The server will start upgrading to Ubuntu version 24.04. Wait until finished, and sometimes you have to answer the questions asked by the Linux system when upgrading. After the upgrade finishes, check the version of Ubuntu, like in the image below:

If during the upgrade process, there is a notification like the picture below:

Could not calculate the upgrade

An unresolvable problem occurred while calculating the upgrade.

Type the command below to see the errors that occurred during the upgrade process:

cat /var/log/dist-upgrade/main.log | grep ERROR

In the log, you have to search for the cause of the error, but actually, you can find the root cause in the notification, like in the image below:

The root cause is in the postgresql-12 package, so I removed the package and then ran the command below to carry out the upgrade process again:

sudo do-release-upgrade

The Ubuntu upgrade process should be completed until it is finished.

Note

When you upgrade Ubuntu, you have to answer the questions from the Ubuntu system, like in the image below:

If you don’t want to be bothered by the questions asked by the Linux system during the upgrade process, then use the command below:

sudo do-release-upgrade -f DistUpgradeViewNonInteractive

References

ubuntu.com
serverpilot.io
jumpcloud.com
askubuntu.com