Skip to content

S3 403 Access Denied: the systematic debug order that actually finds the cause

An S3 403 Access Denied can come from six different layers: IAM, bucket policy, Block Public Access, ownership, KMS, or an SCP. Here is the order to check them in so you find the real one.

S3 returns 403 Access Denied when any one of roughly six evaluation layers denies the request — or when the object does not exist and your caller lacks s3:ListBucket, because S3 deliberately hides missing keys from identities that cannot list. The productive move is not to start editing policies; it is to walk the layers in a fixed order: confirm the calling identity, then the IAM policy, the bucket policy, Block Public Access, object ownership, and finally KMS. One of them is the answer, and the order below is sorted by how often each one turns out to be it.

What’s happening

Every S3 request goes through the same evaluation. An explicit Deny anywhere wins immediately — bucket policy, IAM policy, SCP, permissions boundary, session policy, or VPC endpoint policy. If nothing denies, the caller still needs at least one Allow: from its own IAM identity policy when caller and bucket are in the same account, or from both the caller’s IAM policy and the bucket policy when they are in different accounts. On top of that sit gates that people forget exist: service control policies at the organization level, the policy on the VPC endpoint the traffic happens to route through, Block Public Access overriding any public grant, and — for objects encrypted with SSE-KMS — the KMS key policy, which is evaluated separately and produces the exact same AccessDenied string when it fails.

Two special cases masquerade as permission problems. First, the missing-object case: without s3:ListBucket on the bucket, a GET for a key that does not exist returns 403, not 404. Second, requester-pays buckets return 403 unless the caller sends --request-payer requester.

What to check first

  • Who is actually calling? The CLI profile, the instance role, or the Lambda execution role is frequently not the identity you think you are testing with.
  • Does the object actually exist, with that exact key, in that exact bucket and region? Keys are case-sensitive and a leading slash is part of the key.
  • Is the object SSE-KMS encrypted? If yes, half your suspects are on the KMS side.
  • Does the traffic cross a VPC endpoint or an organization boundary? Both add policies to the evaluation.
  • What changed recently? A rotated role, a new SCP, a tightened bucket policy — check CloudTrail before you check anything else if the access worked yesterday.

How to verify

Establish identity and object facts first:

# Who am I, really?
aws sts get-caller-identity

# Does the object exist and what encryption does it carry?
aws s3api head-object --bucket my-bucket --key path/to/file.pdf
# Look at ServerSideEncryption and SSEKMSKeyId in the output

Then pull the layers one at a time:

aws s3api get-bucket-policy --bucket my-bucket --query Policy --output text
aws s3api get-public-access-block --bucket my-bucket
aws s3api get-bucket-ownership-controls --bucket my-bucket
aws s3api get-bucket-acl --bucket my-bucket

In the console, everything relevant lives under S3 > Buckets > your-bucket > Permissions. For the authoritative answer on which layer denied, CloudTrail is better than guessing — data events for the bucket (if enabled) show the errorCode and, since the 2023 improvements, AccessDenied messages in most services state which policy type denied. The IAM Policy Simulator (IAM > Policy simulator) is useful for same-account cases but does not evaluate bucket policies for cross-account callers, so treat its verdict as partial.

The fix

  1. Wrong identity. get-caller-identity shows a different account or role than expected — fix the profile, the role attachment, or the credential chain before touching any policy. This closes a large share of tickets on its own.
  2. The /* mistake in the IAM policy. Object actions (s3:GetObject, s3:PutObject) need the object ARN arn:aws:s3:::my-bucket/*; s3:ListBucket needs the bucket ARN arn:aws:s3:::my-bucket. A policy with only one of the two produces 403s that look random.
  3. An explicit deny in the bucket policy. Look for "Effect": "Deny" with conditions like aws:SecureTransport, aws:SourceVpce, or aws:PrincipalOrgID — a TLS-only deny blocks plain HTTP callers, a VPC-endpoint condition blocks everything that arrives over the public internet, including your laptop.
  4. KMS, not S3. For SSE-KMS objects the caller needs kms:Decrypt on the key and the key policy must not exclude them. Cross-account reads need the key policy to grant the external account explicitly. The error still says AccessDenied on the S3 call, which is why this one hides so well.
  5. Cross-account object ownership. In buckets that still have ACLs enabled, an object uploaded by another account can be unreadable even by the bucket owner. Set Object Ownership to Bucket owner enforced to end this class of problem permanently.
  6. Block Public Access doing its job. If the access you expected was public or ACL-based, BPA at the bucket or account level overrides it silently. Decide whether the access should exist at all before loosening anything.
  7. An SCP or permissions boundary. If the same policy works in one account and not another, the organization layer is the suspect.
  8. The 403 that is a 404. Grant s3:ListBucket (or test with an identity that has it) and retry — if it becomes NoSuchKey, you were chasing a permissions ghost.

When this is the wrong move

If the “fix” you are reaching for is "Principal": "*", s3:* on all resources, or turning Block Public Access off, stop. Those make the symptom disappear by making the bucket the next breach headline. A 403 during an incident on data you do not understand is also a signal to slow down — confirm what the bucket holds before widening access under pressure.

Common mistakes

  • Testing with admin credentials, seeing it work, and shipping code that runs under a far narrower role.
  • Editing three policies at once, so when the 403 clears nobody knows which change mattered — or which two are now unnecessary grants.
  • Forgetting that an assumed role’s session policy and its identity policy both constrain the session.
  • Blaming the bucket policy when the object is SSE-KMS and the key policy is the real gate.

Prevention

  • Set Object Ownership: Bucket owner enforced on every bucket; ACLs are a legacy layer you do not want in the evaluation.
  • Enable CloudTrail data events on sensitive buckets so 403s come with forensics instead of guesswork.
  • Run IAM Access Analyzer to see external access the policies actually grant, not what you think they grant.
  • Standardize one TLS-only deny statement and test it in staging before it lands in production bucket policies.
  • Keep one documented debug order (this one) in the runbook so nobody starts at step six. The full permission model is documented at https://docs.aws.amazon.com/AmazonS3/ if you need the fine print.

A 403 always has exactly one first cause, and the layers are enumerable — the debugging feels endless only when you probe them in random order. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.