Errors when deleting a «dead» Docker container

Symptoms

A Docker container with a «dead» status cannot be deleted.

Causes

The Docker container appears «busy» with a process to the file system driver.

Solution

Universal method

You can remove all Docker containers with a «dead» status using the following command:

docker rm $(docker ps --all -q -f status=dead)

Devicemapper driver

When attempting to remove a container in Docker with the devicemapper driver, the following error occurs:

Error response from daemon: Cannot destroy container <container_name>: Driver devicemapper failed to remove root filesystem <container_directory_id>: Device is Busy

In this case, simply unmount the container directory in /var/lib/docker/devicemapper/mnt/ with the following command:

umount /var/lib/docker/devicemapper/mnt/<container_directory_id>

Overlay driver

When attempting to remove a container in Docker with the overlay2 driver, the following error occurs:

Error response from daemon: driver "overlay2" failed to remove root filesystem for : remove /var/lib/docker/overlay/<container_directory_id>/merged: device or resource busy

In this situation, find the process using the /var/lib/docker/overlay/<container_directory_id>/merged directory with the following command:

grep <container_directory_id> /proc/*/mountinfo

In the output, at the beginning of the line, find the PIDs of the processes that use the container directory:

/proc/1010/mountinfo:546 30 0:56 / /var/lib/docker/overlay2/<container_directory_id>/merged ...
/proc/1515/mountinfo:546 30 0:56 / /var/lib/docker/overlay2/<container_directory_id>/merged ...
/proc/2020/mountinfo:546 30 0:56 / /var/lib/docker/overlay2/<container_directory_id>/merged ...

Check which services are busy with processes with the following the command:

ps -p <process_pid> -o comm=

If a non-Docker service is listed among the names, it may be the cause of the error: if possible, stop this service, remove the problematic container, and restart the service.