EC2 placement groups explained: cluster, spread, partition — and when they matter

Most EC2 fleets do not need a placement group, and adding one “for performance” without a measured reason mostly buys new failure modes. They matter in three specific situations: cluster when instances must talk to each other with the lowest possible latency and highest throughput (HPC, tightly coupled compute), spread when a handful of critical instances must never share hardware (quorum nodes, HA pairs), and partition when a distributed datastore like Kafka, Cassandra, or HDFS needs to map its replica awareness onto real rack-level fault domains.

What you are deciding

A placement group is an instruction to the EC2 scheduler about how instances may share underlying hardware. By default, AWS places instances wherever capacity is healthy, with no promises about proximity or separation. The three strategies replace that default with a contract:

• Cluster: pack instances close together — same high-bisection-bandwidth network segment inside a single AZ. You get the lowest node-to-node latency and the ability to actually use the high network throughput (and EFA, if the instance type supports it) between members. The price: a shared blast radius, and insufficient capacity errors become a real operational event, because AWS must find contiguous capacity. Launching all instances at once, same type, ideally in one request, is the well-worn advice.
• Spread: the opposite contract — every instance on distinct hardware racks with separate power and network. Hard limit: seven running instances per AZ per group (you can span multiple AZs, seven each). Built for small sets of instances that must not fail together: etcd/ZooKeeper quorums, licence servers, HA pairs that would otherwise secretly share a rack.
• Partition: the middle path for distributed systems. The group is divided into up to seven partitions per AZ, each partition on isolated racks; instances within a partition may share hardware, but partitions never do. Crucially, the topology is visible: instance metadata exposes the partition number, so Kafka can map brokers, and Cassandra its rack awareness, onto real failure boundaries instead of imaginary ones. Partition groups scale to hundreds of instances, unlike spread.

What placement groups do not do: nothing crosses Regions, cluster groups do not cross AZs, and none of them changes what you pay — there is no charge for the group itself; the cost shows up operationally, as reduced scheduling flexibility and capacity risk. The full strategy details are in the EC2 documentation.

What to check first

• Do you have a measured east-west network problem? Cluster groups fix inter-node latency/throughput — if your bottleneck is disk, database, or the internet path, they fix nothing.
• Is the “must not fail together” set seven or fewer per AZ? That is spread territory; larger sets need partition.
• Does your datastore understand racks? Partition groups only pay off when the application maps replicas to partitions (broker.rack, Cassandra snitch, HDFS rack script).
• Are instance types compatible? Cluster groups want current-generation, network-capable types, and mixing types raises the odds of placement failures.
• Can you tolerate InsufficientInstanceCapacity at scale-out time? Cluster and spread constraints make it more likely, especially after stop/start cycles.

How to verify

List existing groups and their strategies:

aws ec2 describe-placement-groups \
  --query 'PlacementGroups[].[GroupName,Strategy,State,PartitionCount]' --output table

See which group and partition each instance landed in:

aws ec2 describe-instances \
  --filters "Name=placement-group-name,Values=kafka-prod" \
  --query 'Reservations[].Instances[].[InstanceId,Placement.PartitionNumber,Placement.AvailabilityZone,State.Name]' \
  --output table

From inside an instance, the partition number is available through instance metadata (IMDSv2), under the placement/partition-number path — which is what rack-aware applications should read at startup. In the console, EC2 > Network & Security > Placement Groups shows groups; the instance’s Details tab shows its group and partition. For cluster groups, prove the benefit with a before/after iperf3 run between two members — if the numbers do not move, the group is decoration.

The approach

1. Default to no placement group. Stateless web/app tiers behind a load balancer get their resilience from multi-AZ Auto Scaling, not placement contracts.

2. Cluster: create the group, then launch everything in one shot. Same instance type, single launch request or a tightly scripted sequence, all in one AZ:

   aws ec2 create-placement-group --group-name hpc-prod --strategy cluster
   aws ec2 run-instances --count 8 --instance-type c7gn.8xlarge \
     --placement GroupName=hpc-prod \
     --image-id ami-0abc1234def567890 --key-name ops

   If you hit capacity errors, stopping and starting all members sometimes relocates the group; growing an old cluster group one instance at a time is the classic failure.

3. Spread: reserve it for the crown jewels. Quorum members, primary/standby pairs. Respect the seven-per-AZ ceiling and span AZs when the workload allows.

4. Partition: size partitions to your replication factor. Three partitions for RF=3 Kafka/Cassandra is the standard shape; wire the application’s rack awareness to the partition number so a rack loss costs one replica, not two.

5. Put the group in the launch template / ASG config, not in tribal memory — ASGs can target a placement group, and that is how the contract survives instance replacement.

6. Plan capacity explicitly for cluster groups: On-Demand Capacity Reservations inside the group turn “hope the rack has room” into a held guarantee, which matters for scheduled HPC runs.

When this is the wrong move

A cluster placement group under an ordinary microservices stack concentrates your fleet onto shared infrastructure for a latency win the application cannot feel — you keep the blast radius and skip the benefit. Spread groups as a general “more resilient” setting break at instance eight and add capacity friction; multi-AZ ASGs already give ordinary tiers better protection. And if you are fighting for single-digit microseconds, check the instance type and EFA support first — placement is only one term in that equation. Mixed-instance, Spot-heavy fleets in particular sit badly with rigid placement contracts (see mixed instances with Spot and On-Demand).

Common mistakes

• Adding instances to a cluster group months later and treating the resulting InsufficientInstanceCapacity as an AWS outage rather than the expected cost of the contract.
• Assuming a partition group protects data by itself. It only creates the fault domains; the datastore must be configured to place replicas across them.
• Exceeding spread limits by “just adding one more,” which fails at launch — discovered during an incident scale-up.
• Believing placement groups span Regions or make cross-AZ traffic free. They do neither; cross-AZ data transfer still bills normally.

Keeping it under control

• Document, next to each group, the measured reason it exists; re-validate the measurement yearly.
• Alarm on launch failures in ASGs that target placement groups — capacity friction shows up there first.
• For partition groups, assert at deploy time that replicas actually land in distinct partitions (a start-up check reading partition metadata beats an audit later).
• Rehearse the “relocate the cluster group” stop/start procedure before you need it under pressure.

Placement groups are a scalpel: cluster for measured latency needs, spread for small must-not-share sets, partition for rack-aware datastores — and the honest default is none. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.