The Hidden Costs of Oversized Images
Many teams overlook image size during the early stages of containerization. A Spring Boot image often weighs 800MB–1.2GB, and even a Go application image can easily exceed 700MB. The problems caused by oversized images go far beyond “wasting a little disk space”:
- Slow pulls and deployment latency: In CI/CD pipelines or elastic scaling scenarios, nodes must pull the image before starting the container. A 1GB image takes over 80 seconds to download on a 100 Mbps internal network, while a 50MB image takes just 4 seconds. For HPA auto-scaling, this extends the failure recovery window.
- Larger attack surface: Base images include many system utilities you never use (
curl,wget,gcc,bash, etc.). Once an attacker gains shell access to a container, these tools become stepping stones for lateral movement. Smaller images mean a narrower attack surface. - Accumulating storage costs: One 1GB image built 5 times a day and retained for 30 days amounts to 150GB. With 10 microservices, that’s 1.5TB. Harbor / Registry storage and backup costs skyrocket accordingly.
- Inefficient build cache: Larger layers load more slowly on cache hits, slowing down the entire CI pipeline.
This article references the official Docker multi-stage build documentation
Multi-stage Build
Why Multi-stage Builds Are Necessary
The fundamental problem with traditional Dockerfiles is that build tools and runtime environments are bundled into the same image.
Take a Go application as an example: compilation requires the go toolchain and gcc, but the runtime only needs a single binary. If you use golang:1.22 as the base image, the final image includes the entire Go SDK (~800MB+), while your application binary might be only 15MB.
Multi-stage builds allow you to define multiple FROM statements in a single Dockerfile. Each FROM starts a new stage, and the final image only retains the content from the last stage.
Multi-stage Build Syntax
# ===== Stage 1: Build stage =====
FROM golang:1.22-alpine AS builder
WORKDIR /app
# Copy dependency files first to leverage layer caching
COPY go.mod go.sum ./
RUN go mod download
# Then copy source code
COPY . .
# Static build with CGO disabled
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o app ./cmd/server
# ===== Stage 2: Runtime stage =====
FROM alpine:3.19
# Install minimal runtime dependencies
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /app
COPY --from=builder /app/app .
ENTRYPOINT ["/app/app"]
Key points:
AS buildernames the stage, which is later referenced viaCOPY --from=builderCGO_ENABLED=0disables C bindings, producing a pure static binary that runs onscratchoralpine-ldflags="-s -w"strips debug info and symbol tables, reducing binary size by another 30%ca-certificatesprovides the root certificate bundle needed for HTTPS API calls
For more details, see the official Docker multi-stage build documentation
Base Image Selection
Choosing the right base image is the first and most impactful step in image slimming.
Comparison of Three Mainstream Options
| Option | Size | Package Manager | Debuggability | Security | Use Case |
|---|---|---|---|---|---|
alpine | ~5MB | apk | Has shell, can install tools | Medium (musl may cause compatibility issues) | General-purpose lightweight image |
distroless | ~20MB | None | No shell, cannot exec | High (no attack tools) | Security-sensitive production |
scratch | 0MB | None | None at all | Highest | Pure static binaries (Go/Rust) |
Alpine
FROM alpine:3.19
RUN apk --no-cache add ca-certificates
COPY app /app
ENTRYPOINT ["/app/app"]
Alpine’s advantages are its small size and available shell for troubleshooting. However, it uses musl libc instead of glibc, which may cause compatibility issues with CGO-dependent programs (e.g., SQLite).
Distroless
Google’s distroless images contain no shell, package manager, or any unnecessary tools—only the minimal runtime required to run the application.
FROM golang:1.22 AS builder
COPY . /src
WORKDIR /src
RUN CGO_ENABLED=0 go build -o /app ./cmd/server
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app /app
USER nonroot:nonroot
ENTRYPOINT ["/app"]
Note: distroless has no shell, so kubectl exec -it <pod> -- sh will fail. For debugging, you can temporarily switch to an alpine image or use the debug image gcr.io/distroless/static-debian12:debug.
Scratch
FROM scratch
COPY --from=builder /app /app
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
ENTRYPOINT ["/app"]
scratch is an empty image with a size of 0. It’s suitable for pure statically compiled Go / Rust applications. You must manually copy CA certificates, otherwise HTTPS requests will fail with x509: certificate signed by unknown authority.
Layer Caching Optimization
Every Docker instruction (RUN, COPY, ADD) creates a new layer. If a layer’s content hasn’t changed, Docker reuses the cache. Once a layer is invalidated, all subsequent layers are rebuilt.
Instruction Ordering Principle
Place instructions that change infrequently at the top, and those that change frequently at the bottom.
# ✗ Wrong: COPY . . first — any source change invalidates go mod download cache
COPY . .
RUN go mod download
RUN go build -o app .
# ✓ Correct: copy dependency files first, then source code
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -o app .
This way, when you only change business code without touching dependencies, the go mod download layer hits the cache and skips downloading.
.dockerignore
.dockerignore works like .gitignore, excluding files that don’t need to be included in the build context via COPY . ..
# .dockerignore
.git
.gitignore
*.md
docker-compose*.yml
Dockerfile*
node_modules
dist
build
.env
.env.*
*.test.go
*_test.go
.dockerignore not only reduces the build context size (speeding up docker build’s context upload) but also prevents sensitive files like .env from being accidentally packaged into the image.
Image Slimming Tools
dive: Layer-by-Layer Image Analysis
dive is a tool that visually analyzes the content of each Docker image layer, making it easy to see which files take up space and which layers are wasteful.
# Install
brew install dive
# Analyze image
dive myapp:latest
dive displays each layer of the image, the files added/removed in that layer, and an efficiency score. If it detects that apt-get install didn’t clean up /var/lib/apt/lists/* in a layer, dive highlights it in red.
# Check image efficiency in CI
dive myapp:latest --ci \
--efficiency 0.9 \
--wasted-bytes 20MB \
--wasted-pct 5
docker-slim
docker-slim automatically trims images through runtime analysis. It starts the container, monitors its behavior, records the files actually used, and then generates a slim image containing only the necessary files.
# Install
brew install docker-slim
# Analyze and slim
docker-slim build --target myapp:latest --tag myapp:slim
# Compare results
docker images myapp
# myapp latest 876MB
# myapp slim 18MB
docker-slim works exceptionally well for static file serving applications, but for dynamically loading applications (e.g., Java apps that load classes via reflection), it may accidentally delete needed files. In such cases, use --include-path to manually specify paths to retain.
Hands-on: Go Application from 876MB to 18MB
Initial Version (876MB)
Many teams start with a Dockerfile like this:
# Dockerfile.v1 — Initial version
FROM golang:1.22
WORKDIR /app
COPY . .
RUN go build -o app ./cmd/server
CMD ["./app"]
$ docker build -t myapp:v1 -f Dockerfile.v1 .
$ docker images myapp:v1
myapp v1 876MB
Problem: Using golang:1.22 (based on Debian) as the runtime image includes the entire Go SDK, Debian toolchain, and debug symbols.
Version 2: Multi-stage Build + Alpine (126MB)
# Dockerfile.v2 — Multi-stage + alpine
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -o app ./cmd/server
FROM alpine:3.19
RUN apk --no-cache add ca-certificates
WORKDIR /app
COPY --from=builder /app/app .
CMD ["./app"]
$ docker images myapp:v2
myapp v2 126MB
Reduced from 876MB to 126MB—an 85% reduction. But there’s still room—the binary still carries debug symbols.
Version 3: Compile Optimization + Scratch (18MB)
# Dockerfile.v3 — Final version
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# -s -w strips debug symbols and DWARF info
# CGO_ENABLED=0 produces a pure static binary
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
go build -ldflags="-s -w" -o app ./cmd/server
FROM scratch
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /app/app /app
ENTRYPOINT ["/app"]
$ docker images myapp:v3
myapp v3 18MB
Optimization Results Comparison
| Version | Base Image | Size | Optimization Method |
|---|---|---|---|
| v1 | golang:1.22 | 876MB | None |
| v2 | golang:alpine → alpine | 126MB | Multi-stage build + alpine |
| v3 | golang:alpine → scratch | 18MB | + CGO disabled + symbol stripping |
# Verify final image with dive
$ dive myapp:v3
Total Image size: 18 MB
Potential wasted space: 0 B
Image efficiency score: 100 %
The final image has a 100% efficiency score with zero wasted layers.
Final Checks Before Pushing
# Check for residual build tools in the image
$ docker run --rm myapp:v3 ls /usr/bin 2>/dev/null || echo "No /usr/bin (scratch image has no filesystem)"
# Check for .env or key file leaks
$ docker run --rm myapp:v3 find / -name "*.env" -o -name "*.key" 2>/dev/null
# Scan image for vulnerabilities
$ trivy image myapp:v3
Summary
Image optimization is not a one-time task—it should be integrated into CI/CD pipelines as a standard check:
- Multi-stage build is the foundation—separating build and runtime environments offers the best ROI
- Choose the right base image—scratch or distroless for Go/Rust, alpine variants for Python/Node
- Optimize layer caching—copy dependency files first, source code second, and use
.dockerignoreto reduce context size - Tune compilation flags—
-ldflags="-s -w"andCGO_ENABLED=0can reduce size by another 30% - Use tools for verification—
divefor layer efficiency,docker-slimfor automatic trimming,trivyfor vulnerability scanning
By optimizing the image from 876MB to 18MB, pull time drops from 70 seconds to 1.5 seconds, storage costs decrease by 97%, and vulnerability count drops from 200+ to 0. This isn’t a nice-to-have—it’s a fundamental SRE practice.
References & Acknowledgments
This article referenced the following materials during writing. We thank the original authors for their contributions:
- official Docker multi-stage build documentation — Docker Inc., referenced for official Docker multi-stage build documentation
- debug image — GitHub, referenced for debug image
- dive — GitHub, referenced for dive
- docker-slim — GitHub, referenced for docker-slim