Skip to content

AWS Updated 2026-07-04

S3 503 Slowdown: request-rate limits and prefix design

S3 503 Slow Down errors mean you outran a prefix's request rate — 3,500 writes or 5,500 reads per second — before S3 finished scaling. How to fix it with retries, prefix design, and ramping.

An S3 503 Slow Down means your request rate hit a partition of the bucket harder than it could serve at that moment — the baseline is at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix, and S3 scales beyond that automatically but not instantly. The immediate fix is exponential backoff with jitter (which the AWS SDKs already do, if you have not fought them); the durable fix is spreading load across more prefixes and ramping up instead of slamming from zero to peak.

What’s happening

S3 stores your keys across internal partitions and each partition sustains a bounded request rate. When traffic to one key range exceeds what its partition serves, S3 returns 503 SlowDown while it repartitions in the background — that scaling takes minutes to tens of minutes, and during a sudden spike you eat 503s until it catches up. There is no per-bucket hard cap: buckets routinely serve millions of requests per second, but only when the load is spread across many prefixes so S3 has already split the key space.

“Prefix” here means the key-name path S3 uses to partition, not the folder icon in the console. Keys sharing a long common leading string tend to land on the same partitions. The classic self-inflicted wound is sequential naming — logs/2026-07-04-00-00-01.gz, -02.gz, -03.gz — where every write lands on the same partition edge. Modern S3 handles date-based naming far better than it did years ago (key hashing is no longer general advice), but at tens of thousands of requests per second against one narrow key range, partition pressure is still real.

Two adjacent throttles get misdiagnosed as S3’s fault. If objects use SSE-KMS, every request adds a KMS call, and KMS has its own per-region requests-per-second quota (varies by region; check yours in Service Quotas) — KMS throttling surfaces as errors on your S3 operations. And Lambda-driven fan-out can throttle on the Lambda side, producing retry storms that then trip S3.

What to check first

  • Is your client retrying with backoff at all? Raw HTTP calls or misconfigured SDKs turn a 30-second scaling event into an outage.
  • What does the key distribution look like for the hot workload — one narrow prefix or many?
  • Did the load appear suddenly? A new batch job, a backfill, a cache flush — spiky arrival is the trigger; steady load at the same volume usually is not.
  • Are the objects SSE-KMS encrypted? Rule KMS throttling in or out early.

How to verify

Enable request metrics if you have not (they are opt-in and billed): S3 > Buckets > your-bucket > Metrics > Request metrics. Then check the 5xx picture:

aws cloudwatch get-metric-statistics \
  --namespace AWS/S3 \
  --metric-name 5xxErrors \
  --dimensions Name=BucketName,Value=my-bucket Name=FilterId,Value=EntireBucket \
  --start-time 2026-07-03T00:00:00Z --end-time 2026-07-04T00:00:00Z \
  --period 300 --statistics Sum

Client-side, count the actual error strings in application logs — SlowDown and 503 — and note which keys they cluster on. If you run S3 server access logging or CloudTrail data events, aggregate by key prefix to find the hot range. For the KMS side:

aws cloudwatch get-metric-statistics \
  --namespace AWS/KMS --metric-name ThrottledCount \
  --dimensions Name=KeyId,Value=1234abcd-12ab-34cd-56ef-1234567890ab \
  --start-time 2026-07-03T00:00:00Z --end-time 2026-07-04T00:00:00Z \
  --period 300 --statistics Sum

The fix

  1. Confirm retries with exponential backoff and jitter. AWS SDKs do this by default (adaptive retry mode is worth enabling for throttle-heavy workloads); if you disabled retries for latency reasons, re-enable them for 503s specifically. A correctly retrying client often rides out a scaling event with nothing worse than elevated p99.
  2. Spread the key space. Introduce high-cardinality path segments early in the key — tenant ID, hash bucket, worker ID: data/7f/tenant-4211/2026-07-04/... rather than data/2026-07-04/... for everything. More distinct prefixes means S3 can parallelize partitioning.
  3. Ramp, don’t slam. For planned bulk work (backfills, migrations), start at a fraction of target throughput and double every few minutes. S3 scales to sustained load gracefully; it is the step function from 0 to 50,000 requests per second on one prefix that hurts.
  4. Parallelize across prefixes deliberately. Batch jobs that process keys in sorted order concentrate fire on one partition after another. Shuffle the work queue or shard workers by hash of the key.
  5. Fix KMS if KMS is the throttle. Enable S3 Bucket Keys on the bucket (Properties > Default encryption) — it collapses most per-object KMS calls and typically cuts KMS request volume dramatically — or request a KMS quota increase in Service Quotas.
  6. For extreme spikes, ask for help. If a launch event needs a known step-change in request rate on a narrow key space, AWS can pre-partition — open a case via https://aws.amazon.com/premiumsupport/ with the bucket, prefixes, and expected rates.

When this is the wrong move

Do not redesign your key schema over a handful of 503s during a one-time backfill — retries plus ramping are enough, and key migrations are expensive and risky. Conversely, if you are fronting the same objects to thousands of readers, the answer is not more S3 prefixes; it is CloudFront in front of the bucket so most GETs never reach S3 at all.

Common mistakes

  • Treating 503s as fatal errors and failing the job, when the correct behavior is documented retry — S3 is telling you to slow down, not that your data is at risk.
  • Adding client-side retries without jitter, so every worker retries in lockstep and the thundering herd re-trips the throttle on schedule.
  • Benchmarking with keys named test-000001 through test-999999 and concluding S3 is slow — the sequential name concentrated the whole test on one partition.
  • Blaming S3 for throttles that CloudWatch shows are KMS ThrottledCount events.

Prevention

  • Alert on the bucket’s 5xxErrors request metric so throttling shows up as a graph, not a customer report — fold it into your CloudWatch alarm baseline.
  • Put high-cardinality segments early in key names for any workload expected to exceed a few thousand requests per second.
  • Enable S3 Bucket Keys wherever SSE-KMS is in use; it is nearly free insurance against KMS throttles.
  • Document a ramp-up procedure for bulk jobs, and make the batch framework’s concurrency a dial, not a constant. The performance guidelines at https://docs.aws.amazon.com/AmazonS3/ are the canonical reference for the per-prefix numbers.

503s are S3’s flow control, not an outage — respected with backoff and answered with prefix diversity, they disappear from the error budget. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.