Debugging Docker local development environments

Well the code works on MY machine

One of the many benefits of a containerisation solution like Docker is that it allows developers to duplicate production runtime environments on their local machines. However, one key difference between local development and production environments is how frequently containers get replaced.

At Simply Business, we deploy code changes dozens of times per day, and as a result production Docker containers are constantly updated. On my local machine, containers get rebuilt only when I think to do it, which admittedly is only when something breaks. And things do break frequently, largely because containers are not designed to be kept running forever. Cache corruption, limits on memory usage, unversioned changes to Docker images, and probably ghosts, have all been frequent sources of bugs that impact only my local machine.

Having a broken development environment immediately blocks all other work and can be incredibly frustrating. Fortunately, I have several go-to tricks that have been useful for fixing many different containerised applications. This guide is intended to help you quickly get up and running again when your local Docker environment breaks, so that you can focus on more important work!

Have you tried turning it off and on again?

When troubleshooting my local environment, I always try the easiest and simplest fixes first. Since each Docker container behaves like an independent virtual machine, restarting each of them often fixes miscellaneous errors with minimal side effects.

How to restart containers

docker restart $(docker ps --quiet)

docker ps lists all the running containers, and the -q or --quiet flag limits the output to only the container IDs. The expression restarts all running containers.

How to restart containers with docker-compose

docker-compose restart

Cattle, not pets

All Docker containers are built from a template (a Docker image), so it's perfectly safe to get rid of your Docker containers since they are designed to be replaced easily. Deleting containers also clears container caches, which can fix errors caused by a cache corruption.

How to remove containers

docker stop $(docker ps --quiet)
docker rm $(docker ps --all --quiet)

The --all or -a flag, when used with docker ps, lists both running and stopped containers. The first expression stops the running containers, and the second removes all stopped containers.

How to remove containers with docker-compose

docker-compose down

Pump up the volume

Volumes are used in Docker for persisted storage. If you do not specify a named volume when starting a Docker container, a new one is created by default. If you're frequently stopping and starting Docker containers, many unused volumes will stick around, taking up disc space on your local machine. Removing them can sometimes fix memory errors.

Be warned: if you're using volumes for persisted storage, removing them will cause you to lose that data.

How to remove volumes

docker volume prune

This expression removes all volumes not mounted to a running container.

Docker images are like onions

What do ogres, onions, and Docker images all have in common? They all have layers! Your local machine will wind up with dangling images any time a layer in a Docker image changes while the image name stays the same. Dangling images don't usually serve any purpose, and they hog disc space. However, if you don't properly specify which version of the image to use when trying to run a container, dangling images can actually cause errors or unexpected behaviour. I find the best practice is to delete dangling images regularly.

How to remove untagged images

docker image prune

This expression removes all untagged (dangling) images.

Going nuclear

Sometimes your development environment is completely borked even after you've tried everything to fix it. When I am ready to give up, I just delete everything and start from scratch.

How to delete absolutely everything

docker stop $(docker ps --quiet)
docker system prune --volumes

The --volumes flag is effectively an alias for docker volume prune. The first expression stops all running containers. The second deletes all containers, volumes, networks, and dangling images. You can optionally add the -a or --all flag to delete all images, not just dangling ones. Caution: You will literally lose everything.

I hope this guide has given you some tips on how to get up and running again when your local Docker environment breaks! Happy debugging!