Skip to content

Docker health checks — beyond "the process is alive"

Health checks tell the orchestrator whether a container can actually serve traffic. How to write good ones, set timing, and surface the signal to load balancers and Compose.

docker ps shows a container as “Up” as long as the main process hasn’t crashed. That’s not a useful signal — a wedged PHP-FPM, a deadlocked Java thread pool, or a database with a corrupted page can all “be up” while serving 500s. A health check turns “the process exists” into “the application can answer.” The orchestrator (Compose, Swarm, Kubernetes-via-the-Dockerfile) reads that signal to decide whether to route traffic, restart, or hold dependents back.

How to verify

Check the health state of running containers and their health history:

docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Image}}"
docker inspect <container> --format '{{json .State.Health}}' | jq

# Last 5 health probe results with exit codes and output
docker inspect <container> --format '{{range .State.Health.Log}}{{.End}} exit={{.ExitCode}}: {{.Output}}{{end}}'

If Status shows Up 5 minutes without (healthy) or (unhealthy), no health check is configured. If health.log shows the same probe failing for hours and the container is still running, the orchestrator isn’t acting on the signal — restart policy and depends_on conditions are doing their own thing.

What’s happening

A Docker health check is a command Docker runs inside the container at a configured interval. Exit code 0 means healthy, anything else means unhealthy. After retries consecutive failures, the container’s Health state flips to unhealthy. After the same number of consecutive successes, it flips back to healthy. start_period (or start_interval) gives slow-starting services time to come up before failures count against them.

The orchestrator does what it wants with this state:

  • Compose depends_on: condition: service_healthy blocks dependent containers from starting until the dependency is healthy.
  • Swarm reroutes traffic away from unhealthy tasks.
  • A reverse proxy on the host can read docker ps and exclude unhealthy containers from upstreams.
  • Docker itself does NOT restart unhealthy containers by default — restart: unless-stopped only kicks in when the container exits. For “restart on unhealthy,” you need additional tooling (autoheal, a sidecar watcher) or move to an orchestrator that handles it natively.

The procedure

  1. Declare the health check in the Dockerfile so it ships with the image:
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
  CMD curl -fsS http://localhost:8080/healthz || exit 1

-f makes curl exit non-zero on HTTP errors (default is to print the body and exit 0). -sS makes it silent except for errors. --interval is how often to probe, --timeout is how long a probe can take, --start-period is the grace window during startup, --retries is the failure threshold.

  1. Override at run time when the Dockerfile’s is wrong for the deployment:
docker run -d \
  --health-cmd "wget -qO- http://localhost:8080/healthz" \
  --health-interval 30s \
  --health-timeout 5s \
  --health-start-period 30s \
  --health-retries 3 \
  myapp:latest
  1. In Compose, use the long form for clarity:
services:
  web:
    image: myapp:latest
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:8080/healthz"]
      interval: 30s
      timeout: 5s
      start_period: 30s
      retries: 3

  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d app"]
      interval: 10s
      timeout: 5s
      start_period: 30s
      retries: 5

  cache:
    image: redis:7
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5

  web2:
    image: myapp:latest
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_healthy

CMD (a list) is the preferred form — no shell, no quoting issues. CMD-SHELL runs through /bin/sh -c for cases where you need shell features.

  1. Write a real health endpoint, not a tautological one. Bad:
GET /healthz → 200 OK "alive"

Good:

GET /healthz → 200 OK if:
  - the request can be served
  - the database connection pool has at least 1 free connection
  - the cache is reachable (with a fast timeout)
  - the disk has writable space
  - the last background job heartbeat is < 5 minutes old
  otherwise → 503 with the failing check named in the body

The right health check answers “should the load balancer send me traffic?” — not “is my main thread looping?”

  1. Split readiness from liveness when it matters:
  • Liveness — “is the process unrecoverable, should I be killed?” Failing this restarts the container.
  • Readiness — “can I take traffic right now?” Failing this removes me from upstreams but doesn’t restart.

Docker’s single HEALTHCHECK collapses both into one signal. Kubernetes splits them. On Docker-alone hosts, the right pattern is usually: have the health check report readiness and rely on restart: unless-stopped plus your alerting for the liveness case.

  1. Surface unhealthy state to operators. The minimum:
# A cron that pages on any unhealthy container
*/2 * * * * docker ps --filter health=unhealthy --format '{{.Names}}' | \
  ifne mail -s "Unhealthy container on $(hostname)" ops@example.com

The ifne (from moreutils) sends mail only if stdin is non-empty.

  1. Restart-on-unhealthy is not built into Docker. Two options:

autoheal sidecar — small container that watches docker ps and restarts unhealthy containers:

services:
  autoheal:
    image: willfarrell/autoheal:latest
    restart: unless-stopped
    environment:
      AUTOHEAL_CONTAINER_LABEL: "autoheal"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  myapp:
    image: myapp:latest
    labels:
      - "autoheal=true"
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:8080/healthz"]
      interval: 30s

Autoheal restarts containers labeled autoheal=true when they go unhealthy. Trade-off: now you have a Docker-socket-mounted container. Treat it like the privileged tool it is.

Swarm/Kubernetes — native restart on unhealthy. The right answer at scale.

Common pitfalls

  • Forgetting start_period on slow-starting services. The container is marked unhealthy during init, depends_on blocks dependents forever, and the autoheal loop never stops.
  • A health check that calls a public endpoint over the network — if DNS is slow or a load balancer is involved, the probe times out for reasons unrelated to the container’s actual health.
  • Using curl in a slim/distroless image where curl isn’t present. Either install curl explicitly or use the binary’s own health command (e.g. pg_isready, redis-cli ping).
  • Health-checking the wrong thing — checking that port 8080 accepts a connection (nc -z localhost 8080) is satisfied by a wedged process holding the socket. Check that the application actually responds correctly.
  • Running a heavy health check (a database query, a full integration probe) every 5 seconds and watching the health check itself become the load problem.

We treat health checks as a deployment artifact, reviewed alongside the application code, with start_period calibrated to real start times. They’re the foundation of every managed operations restart and alerting decision we make.