Some quirks about using Docker in Gitlab pipelines

For most of our projects, we work in GitHub, which means I'm writing CI pipelines to run as GitHub Actions.

And since we use Docker for local development, it's natural to use Docker in our CI pipelines.

But when I started working on a project that uses Gitlab, I found some quirks using Docker that I didn't expect. It required me to take a step back and re-think some things, so I thought I'd share them with you (and future me) as well.

The biggest mental shift for me was between how Gitlab and GitHub handle using Docker images.

In GitHub, their default runners have Docker installed, so you can use Docker images in your GitHub Actions without any extra setup. In Gitlab, it's also trivial to use a Docker image to run a job, but your runner does not have access to the Docker daemon, and all the Docker commands.

If you want access to the Docker toolset, for example to build a new Docker image, you need to use the docker:dind (Docker-in-Docker) image.

But once you go down that path, your runner is now extremely bare. Simple things like curl don't even exist.

In GitHub Actions, I'm used to having a robust runner, like ubuntu-latest, that has a lot of tools pre-installed and also can run Docker images and have the full Docker toolset available.

In Gitlab, I found that if you want to build Docker images, and you go the Docker-in-Docker route, then everything needs to run in Docker. So, for example, if I need to communicate with AWS, instead of using aws-cli on the host runner, I need to use the aws-cli/aws-cli Docker image.

This also means that I need to think about what environment and volumes are available to each of those Docker containers. It took an extra bit of consideration to get it right.

As with so many things in tech, it's important to try to adopt the mindset of the tool you're using. When I was trying to pound the "GitHub-shaped peg into the Gitlab-shaped hole", I was getting frustrated.

But when I stepped back, and adopted the "run everything as a container" mindset, things worked much better.

Here to help,

Joel

P.S. Would you like some help getting your CI pipelines running quickly and watching your teams back? We can help!