Skip to content

Mixed instances policy — running on-demand base plus spot surge without blowing up

Spot is half the cost or better. Using it under an ASG with a sensible on-demand base, diversified across instance families and AZs, takes a half hour of policy work. Here is the configuration that survives an interruption storm.

The web tier runs 12 c6i.large on-demand instances behind an ALB. The bill for that tier alone is meaningful, and the workload is stateless — every request hits Redis for session, every response is reproducible from request data. Spot is the obvious move. The first naive cut, “switch the ASG to 100% spot,” works for a week, then spot capacity for c6i.large in us-east-1a evaporates during a regional surge, every spot interruption notice fires at once, the ALB drains a dozen instances simultaneously, and the application is unavailable for the four minutes it takes the on-demand backfill to launch and warm up. The right way to use spot under an ASG is to keep a meaningful on-demand base, diversify across instance types and AZs so no single capacity pool can cripple the fleet, and pick the allocation strategy that AWS will actually honour during a surge.

What’s happening

The ASG mixed instances policy describes three orthogonal things: the launch template that all instances share (AMI, security groups, IAM, user data — the boot recipe), the instance type overrides that say “any of these types are acceptable,” and the purchase options that say “I want N% on-demand and 100-N% spot, allocate it this way.”

The two allocation strategies that matter today:

  • price-capacity-optimized, the current default and the one to use. AWS allocates spot capacity from pools with both high capacity availability and low price. Capacity-aware first, price-aware second. This is the right choice for almost everything.
  • capacity-optimized, the original capacity-aware allocator. Picks the deepest pools regardless of price. Slightly safer against interruption, slightly more expensive. Use when interruption survivability dominates cost.
  • lowest-price, the legacy allocator. Picks the cheapest pool regardless of depth. Will reliably select the pool everyone else is fleeing to. Do not use for production work.

Diversification is the other lever. A spot fleet that can launch in five instance types across three AZs has fifteen capacity pools to draw from. A spot fleet pinned to one type in one AZ has one pool. When that pool dries up, the entire spot portion of the fleet starts receiving interruption notices on the same minute. Real diversification means picking instance types with similar resource profiles but different families — c6i.large plus c5.large plus c6a.large plus m6i.large (it has more memory but acceptable for a stateless web tier), each available in every AZ the ASG spans.

The on-demand base is your floor. AWS guarantees on-demand capacity; it does not guarantee spot. The base should cover the absolute minimum traffic the workload needs to keep serving, even if every spot instance disappears at once. For a stateless web tier behind an ALB, this is usually 30–40% of total fleet size at typical load — enough that losing every spot instance degrades performance but does not take the site down.

The spot interruption notice is two minutes. AWS sends a CloudWatch event two minutes before terminating a spot instance. Two minutes is enough to drain from the ALB target group (deregistration delay 30 seconds is plenty), flush in-flight requests, and clean up. It is not enough to spin up a replacement on-demand instance from scratch — that is what the on-demand base and the over-provisioning ratio handle.

How to verify

Inspect the current mixed instances configuration:

aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names my-web-prod-asg \
  --query 'AutoScalingGroups[0].MixedInstancesPolicy' \
  --output json

A healthy production policy looks like this:

{
  "LaunchTemplate": {
    "LaunchTemplateSpecification": {
      "LaunchTemplateName": "my-web-prod-lt",
      "Version": "$Default"
    },
    "Overrides": [
      {"InstanceType": "c6i.large"},
      {"InstanceType": "c5.large"},
      {"InstanceType": "c6a.large"},
      {"InstanceType": "m6i.large"},
      {"InstanceType": "m5.large"}
    ]
  },
  "InstancesDistribution": {
    "OnDemandAllocationStrategy": "prioritized",
    "OnDemandBaseCapacity": 4,
    "OnDemandPercentageAboveBaseCapacity": 20,
    "SpotAllocationStrategy": "price-capacity-optimized",
    "SpotInstancePools": 0
  }
}

For a 12-instance fleet, this gives: 4 on-demand always (the base), then 20% of every additional instance is on-demand, 80% is spot. At 12 total: 4 base + 2 on-demand above base + 6 spot. Lose all 6 spot instances at once and the fleet has 6 instances left — half capacity, degraded but serving.

To see how often spot interruptions actually fire on this ASG:

aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2Spot \
  --metric-name InterruptedSpotInstances \
  --dimensions Name=AutoScalingGroupName,Value=my-web-prod-asg \
  --start-time $(date -u -v-30d +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 3600 \
  --statistics Sum \
  --output text

Stable spot pools show maybe one interruption per instance per day under normal conditions, occasionally zero for days at a stretch. A pool under stress will show a sharp cluster of interruptions on a single hour — that is the signature of a regional capacity event, and it is exactly the moment your diversification earns its keep.

The spot interruption event is published to CloudWatch Events / EventBridge:

aws events list-rules \
  --query 'Rules[?contains(EventPattern, `EC2 Spot Instance Interruption Warning`)]'

If the rule does not exist, the ASG is not draining cleanly. Spot instances should have an interruption handler hooked to a lifecycle hook that deregisters from the load balancer before AWS terminates the instance.

The fix

The worked example, on a stateless web tier sized for 12 baseline instances and peaks to 30:

  1. Define the instance type basket. Pick 4–6 types similar in vCPU and memory. For a tier that runs on c6i.large (2 vCPU, 4 GB), the basket is c6i.large, c5.large, c6a.large, m6i.large (4 GB on the memory side is fine), m5.large. Validate the AMI works on all of them. ARM-based types like c6g are a separate basket — do not mix architectures in one policy unless the AMI is multi-arch.

  2. Pick the on-demand base. For a tier where being half-capacity is degraded-but-alive: 30–40% of typical-load fleet size. For a tier that genuinely needs every instance: 60% or more, and consider whether spot is even the right tool. For this example, 4 base on a typical-12 fleet.

  3. Set the on-demand percentage above base. This is the ratio for any scaling above the base. 20% above base is a sensible starting point — it means the scale-out is mostly spot but adds a little on-demand on the way up, so the fleet ratio does not skew too spot-heavy at peak.

  4. Choose the spot allocation strategy. price-capacity-optimized for everything that does not have a specific interruption-survivability requirement. If the workload is genuinely sensitive — long-running jobs that lose meaningful progress on interruption — use capacity-optimized.

  5. Configure the policy. With the ASG already on a launch template:

    aws autoscaling update-auto-scaling-group \
      --auto-scaling-group-name my-web-prod-asg \
      --mixed-instances-policy '{
        "LaunchTemplate": {
          "LaunchTemplateSpecification": {
            "LaunchTemplateName": "my-web-prod-lt",
            "Version": "$Default"
          },
          "Overrides": [
            {"InstanceType": "c6i.large"},
            {"InstanceType": "c5.large"},
            {"InstanceType": "c6a.large"},
            {"InstanceType": "m6i.large"},
            {"InstanceType": "m5.large"}
          ]
        },
        "InstancesDistribution": {
          "OnDemandBaseCapacity": 4,
          "OnDemandPercentageAboveBaseCapacity": 20,
          "SpotAllocationStrategy": "price-capacity-optimized"
        }
      }'
  6. Wire the spot interruption handler. A lifecycle hook on autoscaling:EC2_INSTANCE_TERMINATING is one approach — see the lifecycle hooks article. The simpler approach for ALB-backed fleets is to set the target group’s deregistration_delay.timeout_seconds to 30 and rely on ALB’s connection draining. Either way, verify by manually triggering an interruption with aws ec2 send-spot-fleet-instance-interruption-notification (or wait for a natural one) and watching the access log for clean 200s through the drain.

  7. Test the failure mode. Use aws fis to inject a spot interruption simulation across half the spot instances at once, watch the fleet behaviour, confirm the ALB does not return 5xx during the drain, confirm the ASG launches replacements within minutes. This is what you actually want to know before trusting the policy in production.

The opinionated take on base/spot ratio for a stateless web tier behind an ALB, with reasonably warm AMIs (under 2 minutes to be in service):

  • 30% on-demand base is the right floor. Lose all spot at once and you still serve 30% of traffic on the on-demand portion. ALB target groups will route around the missing instances; latency goes up, requests do not fail.
  • 20% on-demand above base balances cost and resilience on scale-out.
  • Five instance types across three AZs is the minimum diversification — fifteen capacity pools. Less than that and you are betting on a small number of pools staying healthy together.
  • Warmups under 2 minutes. If your AMI takes 5 minutes to bootstrap, the 2-minute interruption notice is not enough to launch a replacement before draining. Use warm pools or raise the on-demand base.

The mixed instances policy is one of the highest-ROI changes available on a stateless tier. The setup is half an hour. The bill saving is meaningful from the first month. The work is making sure the failure mode — a regional capacity event taking out most of your spot — degrades the service gracefully rather than dropping it.


Stack Harbor operates ASG-backed application tiers and cost-capped auto-scaling policies as part of Managed AWS and the Auto-Scaling operating layer. If your scale events are still surprises, start with an infrastructure consult.