EBS spend does not spike; it creeps. Snapshots accumulate daily with no expiry, volumes stay gp2 because nobody revisited a 2020 decision, and deregistered AMIs leave their snapshots behind forever. The remedy has three parts: convert gp2 volumes to gp3 (an online change that usually cuts the per-GB rate), put Data Lifecycle Manager retention policies on every snapshot-producing volume so cleanup stops being a human job, and run one deliberate purge of the historical backlog — including the AMI orphans most cleanups miss.
What’s happening
Two mechanisms drive the creep. On the volume side, gp2 pricing and performance both scale with size, which made over-provisioning a habit: teams sized volumes for IOPS, not capacity. gp3 decouples them — a baseline of 3,000 IOPS and 125 MB/s at any size, extra performance purchasable separately — at a lower per-GB rate. Most gp2 fleets convert with zero downtime and no performance loss; the exception is large gp2 volumes (roughly 1 TB and up) whose scaled IOPS exceeded gp3’s baseline, where you provision the difference explicitly and usually still come out ahead.
On the snapshot side, snapshots are incremental but immortal. Each one stores only blocks changed since the previous snapshot, which keeps the marginal cost low — until years of dailies on a busy volume add up to multiples of the volume’s own size. Deleting a snapshot in the middle of a chain does not necessarily free its full listed size either, because blocks still referenced by other snapshots are retained. And the classic silent leak: deregistering an AMI does not delete its backing snapshots. Accounts that bake AMIs weekly accumulate an orphan graveyard nobody’s cleanup script looks at. The billing structure is per GB-month of stored (compressed, deduplicated) snapshot data, with a separate cheaper archive tier — details in the EBS documentation.
What to check first
- How much of the fleet is still gp2. Filter EC2 > Elastic Block Store > Volumes by Type. Every gp2 GB is a discount you have not clicked yet.
- Whether any lifecycle policy exists. EC2 > Elastic Block Store > Lifecycle Manager — if it is empty and you have thousands of snapshots, they were all made by hand or by scripts with no deletion step.
- Snapshot count and total size trend in Cost Explorer, usage type
EBS:SnapshotUsage, monthly, last 12 months. A straight line up with no dips means nothing is ever deleted. - Orphaned AMI snapshots. Snapshots whose description references an AMI ID that no longer exists.
- Unattached volumes. Covered in the orphan hunt checklist — do that pass first, since deleting a dead volume also ends its future snapshots.
How to verify
# gp2 volumes still in the account
aws ec2 describe-volumes --filters Name=volume-type,Values=gp2 \
--query 'Volumes[].[VolumeId,Size,Iops,State]' --output table
# All snapshots you own, oldest first, with size
aws ec2 describe-snapshots --owner-ids self \
--query 'sort_by(Snapshots,&StartTime)[].[SnapshotId,StartTime,VolumeSize,Description]' \
--output table
# Snapshots created by AMI registration (candidates for orphan check)
aws ec2 describe-snapshots --owner-ids self \
--query 'Snapshots[?contains(Description, `CreateImage`)].[SnapshotId,StartTime,Description]' \
--output table
# AMIs that still exist — anything referenced in a description but absent here is an orphan
aws ec2 describe-images --owners self \
--query 'Images[].[ImageId,Name,CreationDate]' --output table
# Existing DLM policies, if any
aws dlm get-lifecycle-policies
Cross-reference the CreateImage snapshot descriptions against the live AMI list: descriptions name the AMI they were created for, and if that AMI is gone, the snapshot is billing for nothing. For the gp3 conversion, a single call per volume:
aws ec2 modify-volume --volume-id vol-0123456789abcdef0 --volume-type gp3
The runbook
- Convert gp2 to gp3, batched. The modification is online; the volume stays in
optimizingstate for a while with no service interruption. Do a handful of non-critical volumes first, watchVolumeReadOps/latency for a day, then roll through the fleet. For big gp2 volumes, set--iopsand--throughputexplicitly to match what the workload actually used. - Create DLM policies for every snapshot-producing workload. Target volumes by tag, schedule dailies, retain a fixed count or age — for example daily snapshots kept 14 days, weeklies kept 90. Creation and deletion both become automatic, which is the entire point: retention by policy, not by memory.
- Purge the historical backlog once. Oldest first, against a written retention rule (say: nothing older than 12 months unless tagged
retain). Use the Recycle Bin for EBS snapshots with a short retention as your safety net during the purge, so a mistaken deletion is recoverable for a few days. - Deregister dead AMIs, then delete their snapshots. The order matters — a snapshot backing a registered AMI refuses deletion. Deregister first, delete second, and script the pair so they never drift apart again.
- Archive what you must keep but never touch. EBS Snapshot Archive stores full (not incremental) copies at a much lower rate, with a 90-day minimum and a restore measured in hours, not seconds. Right for compliance retention; wrong for anything you might restore in an incident.
- Kill snapshot-producing scripts with no deletion step. Every cron job that calls
create-snapshotmust have a matching retention mechanism or be replaced by a DLM policy.
When this is the wrong move
Do not archive snapshots you rely on for fast recovery — the restore delay from archive is incompatible with incident timelines. Do not purge anything under a legal hold or compliance retention schedule without written sign-off; storage is cheaper than a dispute. And if your snapshots feed a cross-region DR copy process, verify the copy job’s dependencies before deleting sources — some replication setups re-copy from a recent local snapshot.
Common mistakes
- Expecting a snapshot’s full listed size back when deleting mid-chain — referenced blocks stay until nothing points at them.
- Deleting AMIs’ snapshots first and wondering why the AMI still boots — it does not; you now have a broken AMI that fails at launch.
- Setting DLM retention by count on a volume that snapshots hourly, then wondering why 24 x 14 snapshots exist.
- Archiving a snapshot, restoring it two weeks later, and being surprised by the 90-day minimum charge.
Prevention
- Every new volume ships as gp3 in launch templates and IaC modules; gp2 becomes a linting error.
- DLM (or AWS Backup) owns all snapshot creation — humans create snapshots only for named events, tagged with a delete-after date.
- A monthly Cost Explorer glance at
EBS:SnapshotUsage: the line should sawtooth, not climb. - AMI hygiene on a schedule: deregister images older than your rollback horizon and delete their snapshots in the same job.
- Tag
retainandlegal-holdexceptions explicitly so automation can skip them instead of stalling on doubt.
Storage is the quietest line on the bill, which is exactly why it needs policies instead of intentions. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.