S3 404 vs 403: why AWS hides which one you got

If your caller does not have s3:ListBucket on the bucket, S3 returns 403 Access Denied for a missing object instead of 404 Not Found. That is deliberate: a 404 would confirm to an unauthorized caller which keys exist and which do not, so S3 refuses to distinguish. The practical consequence is that half of the “permission errors” teams debug are actually typos in the key, deleted objects, or delete markers — and the fastest way to tell is to ask the question with an identity that can list.

What’s happening

S3’s error logic for a GET on a key that does not exist depends on the caller’s list permission. With s3:ListBucket on the bucket ARN, you get 404 NoSuchKey — S3 trusts you enough to reveal existence, because you could have discovered it by listing anyway. Without it, you get 403 AccessDenied, identical to a genuine permission failure. The design prevents key-space enumeration: an attacker probing backup.sql, .env, secrets.json against a private bucket learns nothing from uniform 403s.

Three more cases blur the line further. A delete marker on a versioned object makes GET return 404 even though the data still exists in older versions. A wrong region or wrong endpoint can produce redirects and errors that clients surface inconsistently. And Block Public Access turns what an anonymous caller might expect to be a 404 into a 403 as well, because the anonymous caller has no list permission by definition.

The reverse confusion also happens: teams see 404, assume “object gone,” and miss that they are querying the wrong bucket, the wrong region’s bucket of the same name pattern, or a key with an invisible difference — URL-encoding, a duplicated prefix like uploads/uploads/file.pdf, a trailing space, or case (Report.PDF vs report.pdf are different objects; keys are case-sensitive).

What to check first

• Does your identity have s3:ListBucket on arn:aws:s3:::the-bucket (the bucket ARN, not /*)? If not, every 403 is ambiguous by design.
• Is the key exactly right? Copy it from a list-objects-v2 result, never from memory or logs that may have decoded/encoded it.
• Is the bucket versioned? If yes, a delete marker may be answering your GET.
• Are you in the right account and region? Same-named buckets across environments (myapp-prod-assets vs myapp-staging-assets) cause a surprising share of these.

How to verify

Resolve the ambiguity with an identity that has both s3:GetObject and s3:ListBucket — an admin session is fine for diagnosis:

# Does the key exist? head-object is the cheapest probe
aws s3api head-object --bucket my-bucket --key uploads/report.pdf

# If that 404s, hunt for near-misses in the prefix
aws s3api list-objects-v2 --bucket my-bucket \
  --prefix uploads/rep --query 'Contents[].Key'

On a versioned bucket, look for delete markers:

aws s3api list-object-versions --bucket my-bucket \
  --prefix uploads/report.pdf \
  --query '{Versions:Versions[].[Key,VersionId,IsLatest],DeleteMarkers:DeleteMarkers[].[Key,VersionId,IsLatest]}'

A delete marker with IsLatest: true means the object was “deleted” but is recoverable. Confirm which identity your failing application actually uses, then test as that identity — role assumption makes this quick:

aws sts get-caller-identity
aws s3api head-object --bucket my-bucket --key uploads/report.pdf --profile app-role

head-object returns 404 vs 403 under the same rules as GET, and CloudTrail data events (if enabled) record the true errorCode server-side. Console check: S3 > Buckets > my-bucket — search the prefix, and toggle Show versions on a versioned bucket.

The fix

1. Diagnose with a listing identity first. One head-object call under an admin or diagnostic role collapses the ambiguity: 404 means the object is not there; 403 under a full-permission identity means a real policy or KMS denial, so switch to the 403 debug order.
2. If it is a missing object, find out why. Check list-object-versions for delete markers, lifecycle rules that may have expired it (Management > Lifecycle rules), and the writing application’s logs — very often the upload failed silently and nothing was ever there.
3. If it is a key mismatch, fix the writer or the reader, not the data. Normalize key construction in one shared function; most double-prefix and encoding bugs come from concatenation logic duplicated in two codebases.
4. Grant s3:ListBucket to application roles that legitimately need real 404s. Applications that branch on “not found” (caches, existence checks) behave far better when S3 can actually say 404. Scope it with a s3:prefix condition if the bucket holds mixed tenants.
5. If anonymous or cross-account callers need clean 404s, think twice. You would have to grant them list rights, which reveals your key space. Usually the better answer is fronting S3 with CloudFront or an API that owns the error semantics.

When this is the wrong move

Do not grant s3:ListBucket broadly just to make error codes prettier — on multi-tenant buckets, list permission across tenants is a data leak (object keys themselves often contain customer names, invoice numbers, email addresses). If your application needs precise existence semantics on sensitive data, put a database in front and treat S3 as blob storage, or use tightly prefix-scoped list conditions.

Common mistakes

• Spending hours in IAM for what a single list-objects-v2 --prefix would have exposed as a typo.
• Testing with your own admin credentials, getting a clean 404, and not noticing the app’s role sees 403 for the same missing key — then “fixing” policies that were never broken.
• Forgetting that HeadObject errors carry no XML body — a bare 403 from a HEAD gives you even less to go on than a GET; reproduce with GET when you need the error code detail.
• Assuming 404 proves deletion when the bucket name, region, or profile was wrong the whole time.

Prevention

• Give production application roles s3:ListBucket with a prefix condition when the app logic branches on existence — deliberate, scoped, documented.
• Enable CloudTrail data events on buckets where you regularly investigate access questions; server-side error codes end the guessing.
• Centralize key construction and log the exact key on every failed S3 call — the full string, not an interpolated summary.
• Keep the S3 error-semantics page from https://docs.aws.amazon.com/AmazonS3/ in the team runbook so the 403-hides-404 behavior is tribal knowledge, not a recurring surprise.

The rule is compact: 403 without list permission is unreadable — resolve it by asking again with an identity that can list, then follow whichever error you actually have. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.