Image by Editor | Midjourney & Canva

Leveraging Docker cache can significantly speed up your builds by reusing layers from previous builds. Let’s learn how to optimize a Dockerfile to make the best use of Docker’s layer caching mechanism.

Prerequisites

Before you start:

• You should have Docker installed. Get Docker if you haven’t already.
• You should be familiar with basic Docker concepts, creating Dockerfiles, and common Docker commands.

How the Docker Build Cache Works

Docker images are built in layers, where each instruction in the Dockerfile creates a new layer. For example, instructions like FROM, RUN, COPY, and ADD each create a new layer in the resulting image.

Docker uses a content-addressable storage mechanism to manage image layers. Each layer is identified by a unique hash that Docker calculates based on the contents of the layer. Docker compares these hashes to determine if it can reuse a layer from the cache.

Building a Docker Image | Image by Author

When Docker builds an image, it goes through each instruction in the Dockerfile and performs a cache lookup to see if it can reuse a previously built layer.

To reuse or build from scratch | Image by Author

The decision to use the cache is based on several factors:

• Base image: If the base image (FROM instruction) has changed, Docker will invalidate the cache for all subsequent layers.
• Instructions: Docker checks the exact content of each instruction. If the instruction is the same as a previously executed one, the cache can be used.
• Files and directories: For instructions that involve files, like COPY and ADD, Docker checks the contents of the files. If the files haven’t changed, the cache can be used.
• Build context: Docker also considers the build context (the files and directories sent to the Docker daemon) when deciding to use the cache.

Understanding Cache Invalidation

Certain changes can invalidate the cache, causing Docker to rebuild the layer from scratch:

• Modification in the Dockerfile: If an instruction in the Dockerfile changes, Docker invalidates the cache for that instruction and all subsequent instructions.
• Changes in source files: If files or directories involved in `COPY` or `ADD` instructions change, Docker invalidates the cache for these layers and subsequent layers.

To sum up, here’s what you need to know about docker build cache:

• Docker builds images layer by layer. If a layer hasn’t changed, Docker can reuse the cached version of that layer.
• If a layer changes, all subsequent layers are rebuilt. Therefore, putting instructions that do not change often (such as the base image, dependency installations, initialization scripts) much earlier in the Dockerfile can help maximize cache hits.

Best Practices to Leverage Docker’s Build Cache

To take advantage of the Docker build cache, you can structure your Dockerfile in a way that maximizes cache hits. Here are some tips:

• Order instructions by frequency of change: Place instructions that change less frequently higher up in the Dockerfile. And place frequently changing instructions, such as COPY or ADD of application code towards the end of the Dockerfile.
• Separate dependencies from application code: Separate instructions that install dependencies from those that copy the source code. This way, dependencies are only reinstalled if they change.

Next, let’s take a couple of examples.

Examples: Dockerfiles That Leverage the Build Cache

1. Here’s an example Dockerfile for setting up a PostgreSQL instance with some initial setup scripts. The example focuses on optimizing layer caching:

# Use the official PostgreSQL image as a base
FROM postgres:latest

# Environment variables for PostgreSQL
ENV POSTGRES_DB=mydatabase
ENV POSTGRES_USER=myuser
ENV POSTGRES_PASSWORD=mypassword

# Set the working directory
WORKDIR /docker-entrypoint-initdb.d

# Copy the initialization SQL scripts
COPY init.sql /docker-entrypoint-initdb.d/

# Expose PostgreSQL port
EXPOSE 5432

The base image layer often doesn’t change frequently. Environment variables are unlikely to change often, so setting them early helps reuse the cache for subsequent layers. Note that we copy the initialization scripts before the application code. This is because copying files that don’t change frequently before those that do helps in leveraging the cache.

2. Here’s another example of a Dockerfile for containerizing a Python app:

# Use the official lightweight Python 3.11-slim image
FROM python:3.11-slim

# Set the working directory
WORKDIR /app

# Install dependencies
COPY requirements.txt requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Copy the contents of the current directory into the container
COPY . .

# Expose the port on which the app runs
EXPOSE 5000

# Run the application
CMD ["python3", "app.py"]

Copying the rest of the application code after installing dependencies ensures that changes to the application code do not invalidate the cache for the dependencies layer. This maximizes the reuse of cached layers, leading to faster builds.

By understanding and leveraging Docker’s caching mechanism, you can structure your Dockerfiles for faster builds and more efficient image creation.

Additional Resources

Learn more about caching at the following links:

Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she’s working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more. Bala also creates engaging resource overviews and coding tutorials.