Advanced Docker: Multi-Stage Builds and Optimization

2025-10-30 · Pavan Y N · docker, advanced, optimization, security

Published: October 30, 2025
Reading Time: 26 minutes
Tags: Docker, Advanced, Optimization, Security, Multi-Stage

# Introduction

While Docker fundamentals get you started with containerization, mastering advanced Docker techniques is essential for building production-ready, secure, and optimized container images. This guide covers multi-stage builds, image optimization, security best practices, and advanced Docker patterns.

# Multi-Stage Builds

## Why Multi-Stage Builds?

Multi-stage builds allow you to use multiple FROM statements in your Dockerfile, enabling you to:

→Separate build and runtime dependencies
→Reduce final image size dramatically
→Improve security by excluding build tools
→Create cleaner, more maintainable Dockerfiles

## Basic Multi-Stage Build

Before (Single Stage):

code

FROM node:20
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
EXPOSE 3000
CMD ["node", "dist/server.js"]

# Result: ~1.2GB image with build tools

After (Multi-Stage):

code

# Build stage
FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

# Production stage
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package*.json ./
EXPOSE 3000
USER node
CMD ["node", "dist/server.js"]

# Result: ~150MB image without build tools

## Advanced Multi-Stage Patterns

1. Separate Dependencies and Build:

code

# Dependencies stage
FROM node:20-alpine AS dependencies
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production && \
    npm cache clean --force

# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && \
    npm run test

# Production stage
FROM node:20-alpine AS production
WORKDIR /app

# Copy only production dependencies
COPY --from=dependencies /app/node_modules ./node_modules

# Copy built application
COPY --from=builder /app/dist ./dist
COPY package*.json ./

# Security: Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001 && \
    chown -R nodejs:nodejs /app

USER nodejs

EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD node healthcheck.js || exit 1

CMD ["node", "dist/server.js"]

2. Go Application with Scratch Base:

code

# Build stage
FROM golang:1.21-alpine AS builder

# Install build dependencies
RUN apk add --no-cache git ca-certificates

WORKDIR /app

# Copy go mod files
COPY go.mod go.sum ./
RUN go mod download

# Copy source code
COPY . .

# Build with optimizations
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
    -ldflags='-w -s -extldflags "-static"' \
    -a -installsuffix cgo \
    -o /app/server .

# Production stage - using scratch for minimal image
FROM scratch

# Copy CA certificates for HTTPS
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy binary
COPY --from=builder /app/server /server

# Copy static files if needed
COPY --from=builder /app/static /static

EXPOSE 8080

ENTRYPOINT ["/server"]

# Result: ~10MB image!

3. Python Application with Poetry:

code

# Build stage
FROM python:3.11-slim AS builder

# Install poetry
RUN pip install poetry==1.6.0

WORKDIR /app

# Copy dependency files
COPY pyproject.toml poetry.lock ./

# Install dependencies to virtual environment
RUN poetry config virtualenvs.in-project true && \
    poetry install --no-dev --no-interaction --no-ansi

# Copy application code
COPY . .

# Production stage
FROM python:3.11-slim AS production

WORKDIR /app

# Copy virtual environment from builder
COPY --from=builder /app/.venv ./.venv

# Copy application code
COPY --from=builder /app/src ./src
COPY --from=builder /app/config ./config

# Create non-root user
RUN useradd -m -u 1001 appuser && \
    chown -R appuser:appuser /app

USER appuser

# Activate virtual environment
ENV PATH="/app/.venv/bin:$PATH"

EXPOSE 8000

CMD ["python", "-m", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]

# Image Optimization Techniques

## 1. Layer Caching Optimization

code

# Bad: Changes to code invalidate dependency cache
FROM node:20-alpine
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build

# Good: Separate dependency installation from code copy
FROM node:20-alpine
WORKDIR /app

# Copy only dependency files first
COPY package*.json ./
RUN npm ci

# Copy code after dependencies are cached
COPY . .
RUN npm run build

## 2. Minimize Layer Count

code

# Bad: Multiple RUN commands create multiple layers
FROM alpine:3.18
RUN apk add --no-cache curl
RUN apk add --no-cache git
RUN apk add --no-cache vim
RUN rm -rf /var/cache/apk/*

# Good: Combine commands into single layer
FROM alpine:3.18
RUN apk add --no-cache \
    curl \
    git \
    vim && \
    rm -rf /var/cache/apk/*

## 3. Use .dockerignore

code

# .dockerignore
node_modules
npm-debug.log
.git
.gitignore
.env
.env.local
*.md
.vscode
.idea
coverage
.nyc_output
dist
build
*.log
.DS_Store
Dockerfile
docker-compose.yml
.dockerignore

## 4. Choose Appropriate Base Images

code

# Comparison of base images for Node.js app

# Full image: ~1.1GB
FROM node:20

# Slim image: ~250MB
FROM node:20-slim

# Alpine image: ~180MB (recommended)
FROM node:20-alpine

# Distroless: ~150MB (most secure)
FROM gcr.io/distroless/nodejs20-debian11

## 5. Remove Unnecessary Files

code

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && \
    # Remove development dependencies
    npm prune --production && \
    # Remove unnecessary files
    rm -rf src tests docs *.md .git

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package*.json ./

# Security Best Practices

## 1. Use Non-Root User

code

FROM node:20-alpine

WORKDIR /app

# Copy application files
COPY --chown=node:node . .

# Install dependencies
RUN npm ci --only=production

# Switch to non-root user
USER node

EXPOSE 3000

CMD ["node", "server.js"]

## 2. Scan for Vulnerabilities

code

# Using Trivy
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy:latest image myapp:latest

# Using Snyk
snyk container test myapp:latest

# Using Docker Scout
docker scout cves myapp:latest

Dockerfile with Security Scanning:

code

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Security scan stage
FROM aquasec/trivy:latest AS security-scan
COPY --from=builder /app /scan
RUN trivy fs --severity HIGH,CRITICAL --exit-code 1 /scan

# Production stage
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
USER node
CMD ["node", "dist/server.js"]

## 3. Use Secrets Securely

code

# Bad: Secrets in environment variables
FROM node:20-alpine
ENV DATABASE_PASSWORD=mysecretpassword
CMD ["node", "server.js"]

# Good: Use Docker secrets or build arguments
FROM node:20-alpine
WORKDIR /app
COPY . .
# Secrets passed at runtime, not baked into image
CMD ["node", "server.js"]

Using Docker Secrets:

code

# Create secret
echo "mysecretpassword" | docker secret create db_password -

# Use in service
docker service create \
  --name myapp \
  --secret db_password \
  myapp:latest

Using BuildKit Secrets:

code

# syntax=docker/dockerfile:1.4
FROM node:20-alpine
WORKDIR /app

# Mount secret during build (not stored in image)
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci

COPY . .
RUN npm run build

code

# Build with secret
docker build --secret id=npmrc,src=$HOME/.npmrc -t myapp:latest .

## 4. Read-Only Root Filesystem

code

FROM node:20-alpine
WORKDIR /app

# Create writable directories
RUN mkdir -p /app/tmp /app/logs && \
    chown -R node:node /app

COPY --chown=node:node . .
RUN npm ci --only=production

USER node

# Application runs with read-only root filesystem
# Only /app/tmp and /app/logs are writable
CMD ["node", "server.js"]

Docker Compose with Security:

code

version: '3.8'

services:
  app:
    image: myapp:latest
    read_only: true
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
    tmpfs:
      - /tmp
      - /app/tmp
    volumes:
      - logs:/app/logs:rw
    user: "1001:1001"

volumes:
  logs:

# Advanced Docker Compose Patterns

## 1. Development vs Production Configurations

docker-compose.yml (base):

code

version: '3.8'

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - NODE_ENV=${NODE_ENV:-production}
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=${DB_NAME}
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes

docker-compose.dev.yml (development overrides):

code

version: '3.8'

services:
  app:
    build:
      target: development
    volumes:
      - .:/app
      - /app/node_modules
    ports:
      - "3000:3000"
      - "9229:9229"  # Debug port
    command: npm run dev
    environment:
      - NODE_ENV=development
      - DEBUG=*

  db:
    ports:
      - "5432:5432"
    volumes:
      - postgres-data:/var/lib/postgresql/data

  redis:
    ports:
      - "6379:6379"

volumes:
  postgres-data:

docker-compose.prod.yml (production overrides):

code

version: '3.8'

services:
  app:
    build:
      target: production
    restart: always
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '1'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  db:
    restart: always
    volumes:
      - postgres-data:/var/lib/postgresql/data
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G

  redis:
    restart: always
    volumes:
      - redis-data:/data

volumes:
  postgres-data:
  redis-data:

Usage:

code

# Development
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

## 2. Health Checks and Dependencies

code

version: '3.8'

services:
  app:
    image: myapp:latest
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
      migration:
        condition: service_completed_successfully
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  migration:
    image: myapp:latest
    command: npm run migrate
    depends_on:
      db:
        condition: service_healthy
    restart: on-failure

  db:
    image: postgres:15-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - postgres-data:/var/lib/postgresql/data

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3

volumes:
  postgres-data:

# Docker Networking Advanced

## 1. Custom Networks

code

version: '3.8'

services:
  frontend:
    image: frontend:latest
    networks:
      - frontend-network
      - backend-network

  backend:
    image: backend:latest
    networks:
      - backend-network
      - database-network

  database:
    image: postgres:15-alpine
    networks:
      - database-network

networks:
  frontend-network:
    driver: bridge
  backend-network:
    driver: bridge
  database-network:
    driver: bridge
    internal: true  # No external access

## 2. Network Aliases

code

version: '3.8'

services:
  app:
    image: myapp:latest
    networks:
      app-network:
        aliases:
          - api.local
          - app.local

  nginx:
    image: nginx:alpine
    networks:
      - app-network
    # Can access app via api.local or app.local

networks:
  app-network:

# STAR Approach: Docker Optimization

Situation: Company's Docker images were 2-3GB each, causing slow deployments, high storage costs, and security vulnerabilities.

Task: Optimize Docker images to reduce size, improve security, and accelerate deployment times.

Action:

•
Image Analysis:
- →Audited all Dockerfiles across 50+ services
- →Identified common inefficiencies
- →Measured baseline metrics (size, build time, vulnerabilities)
•
Multi-Stage Build Implementation:
- →Converted all Dockerfiles to multi-stage builds
- →Separated build and runtime dependencies
- →Used alpine and distroless base images
•
Security Hardening:
- →Implemented non-root users in all images
- →Added vulnerability scanning to CI/CD
- →Removed unnecessary packages and files
- →Implemented read-only root filesystems
•
Build Optimization:
- →Optimized layer caching
- →Created comprehensive .dockerignore files
- →Implemented BuildKit for faster builds
- →Set up Docker layer caching in CI/CD
•
Standardization:
- →Created base image templates
- →Documented best practices
- →Implemented automated linting with Hadolint
- →Conducted team training

Result:

→Average image size reduced from 2.5GB to 200MB (92% reduction)
→Build time decreased from 15 minutes to 3 minutes
→Deployment time reduced from 10 minutes to 2 minutes
→Storage costs reduced by 85%
→Security vulnerabilities decreased by 90%
→Docker Hub bandwidth costs reduced by 80%

# Common Issues & Troubleshooting

## Issue 1: Large Image Sizes

Diagnosis:

code

# Analyze image layers
docker history myapp:latest

# Detailed analysis with dive
docker run --rm -it \
  -v /var/run/docker.sock:/var/run/docker.sock \
  wagoodman/dive:latest myapp:latest

Solutions:

→Use multi-stage builds
→Switch to alpine or distroless base images
→Remove build dependencies
→Combine RUN commands
→Use .dockerignore

## Issue 2: Slow Build Times

Solutions:

code

# Enable BuildKit
# export DOCKER_BUILDKIT=1

# Use build cache
docker build --cache-from myapp:latest -t myapp:new .

# Parallel builds
docker build --build-arg BUILDKIT_INLINE_CACHE=1 .

## Issue 3: Permission Issues

code

# Fix: Set correct ownership
FROM node:20-alpine
WORKDIR /app
COPY --chown=node:node . .
RUN npm ci
USER node
CMD ["node", "server.js"]

# Best Practices Summary

## 1. Image Building

→Use multi-stage builds
→Choose minimal base images
→Optimize layer caching
→Use .dockerignore
→Combine RUN commands

## 2. Security

→Run as non-root user
→Scan for vulnerabilities
→Use secrets management
→Keep base images updated
→Implement read-only filesystems

## 3. Performance

→Minimize image size
→Optimize build cache
→Use BuildKit
→Implement health checks
→Monitor resource usage

## 4. Maintainability

→Document Dockerfiles
→Use consistent patterns
→Version images properly
→Automate builds
→Test images before deployment

# Conclusion

Mastering advanced Docker techniques is crucial for building production-ready containerized applications. By implementing multi-stage builds, optimizing images, and following security best practices, you can create efficient, secure, and maintainable container images that scale with your organization's needs.

# Additional Resources

Ready to optimize your Docker images? Start with multi-stage builds and gradually implement more advanced patterns!