Skip to content

Linkerd traffic policy and canary releases with HTTPRoute

Run safe canary releases on Linkerd using HTTPRoute weighted routing and ServiceProfile, with metric-driven promotion via Flagger.

You have Linkerd installed and the team wants progressive delivery — shift 5% of traffic to a new version, observe error rate and latency, promote if healthy, rollback otherwise. Linkerd’s built-in weighted routing via Gateway API HTTPRoute does the traffic-split half. Flagger does the metric-driven promotion half. This article is the operating procedure we follow: set up the HTTPRoute manually first to learn the semantics, then layer Flagger on top so the on-call doesn’t have to make the promote/rollback call at 03:00.

How to verify

The Gateway API CRDs must exist and Linkerd’s HTTPRoute controller must be running:

kubectl get crd | grep gateway.networking.k8s.io
kubectl -n linkerd get pods -l linkerd.io/control-plane-component=destination

A canary HTTPRoute should split traffic by weight when listed:

kubectl -n app get httproute checkout-canary -o yaml | grep -A2 backendRefs

Live traffic splits should be observable in linkerd viz:

linkerd -n app viz stat deploy -o wide
linkerd -n app viz routes svc/checkout

Flagger canaries should appear in their own CRD:

kubectl -n app get canary

What’s happening

Linkerd 2.14+ adopts the Kubernetes Gateway API for L7 traffic policy. An HTTPRoute resource attached to a parent Service splits incoming traffic across multiple backendRefs by weight. The proxy on the client side reads the route, hashes per-request, and picks a backend — there is no L7 gateway in the path, the policy is applied inline by the calling pod’s linkerd-proxy. This is fundamentally different from Istio’s VirtualService (which usually runs at the ingress gateway or per-namespace), and it means weighted routing has the same latency profile as the underlying call.

Flagger sits on top. It watches a Canary CRD, manages two deployments (primary and canary), updates an HTTPRoute weight every interval, and queries Prometheus for success rate and request duration. If both stay above thresholds, weight grows step by step until 100% canary, then the primary is updated and weight drops to 0. If either threshold is breached, Flagger writes weight back to 0 (rollback) and emits an event. The whole loop is reconciliation, not a one-shot pipeline — which means an in-flight canary survives a Flagger restart.

The procedure

  1. Install Gateway API CRDs if not already present:

    kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml
  2. Define a manual canary HTTPRoute to learn the moving parts. Create two deployments, checkout-v1 and checkout-v2, both backing services with the same selector + a version label:

    apiVersion: gateway.networking.k8s.io/v1
    kind: HTTPRoute
    metadata:
      name: checkout-canary
      namespace: app
    spec:
      parentRefs:
        - name: checkout
          kind: Service
          group: ""
      rules:
        - backendRefs:
            - name: checkout-v1
              port: 80
              weight: 90
            - name: checkout-v2
              port: 80
              weight: 10

    Send traffic, watch linkerd viz stat deploy checkout-v1 checkout-v2. You should see roughly 90/10. Adjust weights and watch the split shift in real time.

  3. Install Flagger via Helm into its own namespace:

    helm repo add flagger https://flagger.app
    helm upgrade -i flagger flagger/flagger \
      --namespace flagger \
      --create-namespace \
      --set meshProvider=linkerd \
      --set metricsServer=http://prometheus.linkerd-viz:9090

    Point metricsServer at whichever Prometheus actually scrapes Linkerd metrics — that’s likely your central Prometheus, not the embedded viz one.

  4. Define a Canary resource. Flagger expects a single Deployment to manage (it builds a -primary clone automatically):

    apiVersion: flagger.app/v1beta1
    kind: Canary
    metadata:
      name: checkout
      namespace: app
    spec:
      provider: linkerd
      targetRef:
        apiVersion: apps/v1
        kind: Deployment
        name: checkout
      service:
        port: 80
      analysis:
        interval: 1m
        threshold: 5
        maxWeight: 50
        stepWeight: 5
        metrics:
          - name: request-success-rate
            thresholdRange:
              min: 99
            interval: 1m
          - name: request-duration
            thresholdRange:
              max: 500
            interval: 1m
        webhooks:
          - name: load-test
            url: http://flagger-loadtester.flagger/
            timeout: 5s
            metadata:
              cmd: "hey -z 1m -q 5 -c 2 http://checkout.app/api"
  5. Trigger a canary by pushing a new image tag. Update the Deployment’s image; Flagger detects the change, scales up the canary, starts shifting traffic 5% per minute, and checks metrics. The current weight is visible at kubectl -n app get canary checkout -o jsonpath='{.status.canaryWeight}'.

  6. Watch a rollback in pre-prod. Deploy a deliberately broken version to verify the rollback path before relying on it in prod. The canary should fail one of the metric thresholds, weight should drop to zero, and a Kubernetes Event should record Halt advancement no values found for metric request-success-rate or similar.

Operational notes

  • Metric scrape lag delays decisions. If Prometheus scrapes Linkerd every 30s and Flagger evaluates every 1m, the first evaluation sees mostly empty data and may report success rate 100% on near-zero traffic. Issue load via the webhooks.load-test block or with k6 to make the analysis meaningful.
  • maxWeight: 50 plus shadowing is safer than 100. A blue/green or mirror mode lets you observe production traffic on the canary without committing user requests to it. Switch to weighted only after shadowing is clean.
  • Flagger requires the canary to have its own metric. If your app emits its own request_duration_seconds Prometheus metric, point Flagger at that instead of Linkerd’s response_latency_ms_bucket — application-specific SLIs always beat infra-level ones for promotion gating.
  • HTTPRoute attaches to a Service, but workloads see the Service IP through their sidecar. A non-meshed caller bypasses the split entirely — every request goes wherever DNS resolves first. Confirm callers are meshed before relying on a split.
  • Canary deletes itself when finalised. When the Canary CRD reports Succeeded, the Deployment is updated and Flagger collapses the canary. Don’t keep the Canary resource around for inspection — Flagger will simply not find it next time you push.

Stack Harbor runs progressive delivery on Linkerd-meshed clusters for clients who need predictable release safety without paying for a full Argo Rollouts/Istio stack — including the Prometheus metric design, the load-test harness, and the rollback rehearsals. See how we operate Kubernetes platforms via managed operations.