Skip to content

Argo Rollouts: canary and blue/green deployments on Kubernetes

Install Argo Rollouts and define canary and blue/green strategies with traffic-managed routing via Istio, ingress-nginx, or Gateway API.

You are picking Argo Rollouts because the team needs more control over deployments than what Deployment offers — the standard rolling update is fine for most apps, but a payments service needs an automatic rollback gate on real-time metrics. Argo Rollouts replaces Deployment with a Rollout resource that supports canary and blue/green strategies, integrates with Prometheus and Datadog for analysis, and orchestrates traffic through a mesh, ingress, or Gateway API.

How to verify

The Argo Rollouts controller should be running in its own namespace:

kubectl -n argo-rollouts get deploy argo-rollouts

A Rollout should appear as a top-level resource and its status should report phases:

kubectl -n app get rollout
kubectl -n app argo rollouts get rollout checkout --watch

An AnalysisRun should be observable during a canary:

kubectl -n app get analysisrun
kubectl -n app describe analysisrun <name>

The dashboard (optional) gives a visual view of in-flight rollouts:

kubectl argo rollouts dashboard

What’s happening

A Rollout resource has the same pod template as a Deployment but a different update strategy. With canary, you define steps — setWeight, pause, analysis, experiment, setHeaderRoute — and the controller advances through them in order. With blue/green, you keep the active version under one Service and the preview under another; promotion swaps the Service selector atomically.

The traffic routing is delegated. By default the controller adjusts replica counts on the canary and stable ReplicaSets, and the standard Service load-balances across all matching pods, which gives a coarse weight matching the replica ratio. The interesting modes use a traffic router — Istio VirtualService weights, ingress-nginx canary annotations, Gateway API HTTPRoute backendRefs, ALB target-group weights, or AppMesh listener weights — so weight is enforced at the L7 layer regardless of pod count.

The other half is AnalysisTemplate: a Prometheus or Datadog query (and threshold) that the controller runs during a canary step. Pass-through means promote; fail means rollback.

The procedure

  1. Install the controller and the kubectl plugin.

    kubectl create namespace argo-rollouts
    kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
    
    curl -LO https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-linux-amd64
    sudo install -m 755 kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts
  2. Convert a Deployment to a Rollout. The template block is identical; the kind and strategy change:

    apiVersion: argoproj.io/v1alpha1
    kind: Rollout
    metadata:
      name: checkout
      namespace: app
    spec:
      replicas: 6
      selector:
        matchLabels: { app: checkout }
      template:
        metadata:
          labels: { app: checkout }
        spec:
          containers:
            - name: checkout
              image: registry.example.com/checkout:v1
              ports: [{ containerPort: 8080 }]
      strategy:
        canary:
          canaryService: checkout-canary
          stableService: checkout-stable
          trafficRouting:
            istio:
              virtualService:
                name: checkout
                routes: [primary]
          steps:
            - setWeight: 5
            - pause: { duration: 2m }
            - analysis:
                templates:
                  - templateName: success-rate
            - setWeight: 25
            - pause: { duration: 5m }
            - setWeight: 50
            - pause: { duration: 5m }
            - setWeight: 100
  3. Define an AnalysisTemplate. This is the metrics gate the controller will consult:

    apiVersion: argoproj.io/v1alpha1
    kind: AnalysisTemplate
    metadata:
      name: success-rate
      namespace: app
    spec:
      args:
        - name: service-name
          value: checkout-canary.app.svc.cluster.local
      metrics:
        - name: success-rate
          interval: 30s
          count: 5
          successCondition: result[0] >= 0.99
          failureLimit: 2
          provider:
            prometheus:
              address: http://prometheus.monitoring:9090
              query: |
                sum(rate(istio_requests_total{
                  destination_service="{{args.service-name}}",
                  response_code!~"5.."}[1m]))
                /
                sum(rate(istio_requests_total{
                  destination_service="{{args.service-name}}"}[1m]))

    The controller runs the query every 30 seconds for five iterations; two failures abort and rollback.

  4. Wire two Services and the traffic router. With Istio, the canary and stable Services point at the same selector but the VirtualService has two routes weighted by the Rollout. With ingress-nginx, you use nginx.ingress.kubernetes.io/canary: "true" and canary-weight on a sibling Ingress.

    apiVersion: networking.istio.io/v1
    kind: VirtualService
    metadata:
      name: checkout
      namespace: app
    spec:
      hosts: [checkout]
      http:
        - name: primary
          route:
            - destination: { host: checkout-stable }
              weight: 100
            - destination: { host: checkout-canary }
              weight: 0

    The Rollout controller mutates the weights — you do not edit them by hand.

  5. Trigger a rollout by updating the image:

    kubectl -n app set image rollout/checkout checkout=registry.example.com/checkout:v2

    Watch progression:

    kubectl argo rollouts get rollout checkout -n app --watch
  6. Test the rollback path before relying on it. Deploy a deliberately-broken image; the AnalysisTemplate should fail and the controller should set the canary weight to zero. Time how long that takes — it is your real “time to roll back”.

Operational notes

  • Replica counts during canary can swing. With dynamicStableScale: true, the stable scales down as canary scales up, halving steady-state cost. Without it, you temporarily run 1.x the original replica count. Choose deliberately.
  • AnalysisRun creates pods if your AnalysisTemplate is a Job-based provider. Those pods cost something. For high-frequency canaries, keep providers as direct queries (Prometheus, web) rather than Job-based.
  • Argo Rollouts and HPA conflict. HPA scales the Rollout’s replica count; the canary strategy also adjusts it. The controller correctly handles HPA targeting but only if the HPA references the Rollout, not the underlying ReplicaSets. Use metadata.name of the Rollout in the HPA’s scaleTargetRef.
  • Traffic-router config drift is silent. If something else (a human, another Argo CD app) edits the VirtualService weights between rollouts, the next rollout will write its own weights and obliterate the manual change. Treat the routing CRD as fully controller-managed.
  • The dashboard is unauthenticated. kubectl argo rollouts dashboard is a local-only debugging tool; do not expose it via an Ingress.

Stack Harbor designs the rollout strategy and analysis templates for clients running progressive delivery — choosing canary vs blue/green per workload, defining the SLI used as the promotion gate, and wiring the traffic router to whichever mesh or ingress the cluster runs. See how we operate that within managed Kubernetes operations.