S3 HeadObject 403: prove the key before widening access

An S3 HeadObject 403 does not prove that the object exists or that the caller needs broader permissions. For a missing key, S3 can return 403 when the requester lacks s3:ListBucket. Establish the exact bucket, key and calling identity, then compare independent evidence of existence with the request’s authorization context before changing a policy. ^[1]

Scenario: an export worker can read objects under exports/, but an attachment lookup fails with 403. The application constructed exports/Invoice-42.json; the upload record may identify exports/invoice-42.json. The worker intentionally cannot list the bucket. Adding broad access might change the error while leaving the key construction bug intact.

This is an illustrative investigation for general purpose S3 buckets. It is not a live AWS incident or a claim that the commands below were run against a customer account. Directory buckets, access points and a CDN in front of S3 have additional endpoint and authorization behavior; establish which path is involved before applying this runbook. ^[1]

What can a HEAD response actually tell you?#

HeadObject retrieves object metadata without returning the object body. Successful metadata access is useful evidence for that exact request, but it is not a test of every operation the application will perform later. A failed HEAD also gives less information than engineers often assume. ^[1][2]

The missing-key distinction is deliberate: when the caller lacks bucket-list permission, the service need not disclose nonexistence through a 404. A 403 therefore leaves at least two important possibilities open: the exact key is absent under this visibility policy, or some part of authorization rejects the request. It can also be a different request context from the one you intended to test. ^[1]

Record the observed result without upgrading it into a conclusion. “HEAD denied for this identity and key” is accurate. “The object exists but IAM is broken” is not established. The distinction matters because the second sentence sends the incident toward policy changes before anyone inspects the object name.

Likewise, a 200 for a reference object does not establish access to all objects in the bucket. The failing key can be outside the permitted prefix, use a different encryption key, refer to a version or differ in ownership context. Use reference results to narrow hypotheses, then inspect the remaining object-specific differences. ^[1]

Freeze the request before editing a policy#

Capture the application’s effective identity and request tuple. In the worker environment, inspect which role or user its credentials identify. A successful local CLI test with a developer profile is a different experiment. GetCallerIdentity reports the identity represented by the credentials used for that call. ^[5]

# Run in the same execution context as the failing worker.
# Substitute real values privately; examples are fictional.
aws sts get-caller-identity --output json
aws s3api head-object \
  --bucket example-export-bucket \
  --key 'exports/Invoice-42.json' \
  --expected-bucket-owner '123456789012'

Record the region and endpoint selection as well as the bucket owner you expected. The owner condition is a guard against investigating the wrong bucket context; a mismatch can itself reject the request. Keep it identical between comparison requests and verify its value with the resource owner.

Log the key as structured data, not only as a rendered hyperlink. An object key is not automatically the same as the URL path a browser displays. Spaces, case, Unicode, literal percent signs and decoding rules can change what a path-building layer sends. Compare the upload record’s exact stored key with the SDK input before chasing permission differences.

Avoid copying credentials, signed URLs or full debug output into the editorial record or a public issue. The useful diagnostic record is identity ARN, operation, bucket owner, key or redacted key fingerprint, version selection, relevant request options, time and service request identifier. Keep sensitive object names and payload metadata in the appropriate restricted incident system.

Use a trusted manifest to break the ambiguity#

For the scenario, the upload path should persist the actual bucket and object key returned by its own application logic. If versioning matters, preserve the version identifier too. The worker should consume that reference rather than recreate a filename from a customer-facing label.

Suppose the trusted upload record says exports/invoice-42.json. Compare that exact value with the failed input. This is a concrete difference you can fix in the application without granting the worker a new ability to enumerate unrelated objects. Then retry the intended metadata request through the same runtime identity.

The upload record can outlive the object it names. It can be stale if another actor deletes the object, a lifecycle rule expires it, or the upload did not complete as the application expected. Record whether the manifest represents a completed upload and whether version or deletion events can invalidate it. An independent owner check can establish the current object/version state using already authorized diagnostic access.

Do not use the failing worker’s own HEAD as the independent existence check. That repeats the ambiguous experiment. Ask the bucket owner to verify the exact reference or use an existing inventory/administrative view with known visibility. Preserve the worker’s intended narrow role while resolving the question.

The application repair is then explicit: store the authoritative object reference, pass it unchanged, and reject an incomplete reference. If the product allows renaming a display label, that change must not silently mutate the storage key. This is a data-contract decision as much as an IAM investigation.

Which comparison requests are worth making?#

Choose a known existing reference object under the same intended prefix and with comparable encryption and ownership. Compare it with the failing request under the same worker identity. If necessary, have the owner verify a deliberately absent synthetic key in a disposable test prefix. Do not create or inspect another tenant’s object to make the comparison convenient.

Choose the reference object so you can explain every relevant difference. Testing an unencrypted public sample against an encrypted private export changes too many variables. Testing from a laptop outside the worker’s VPC can change endpoint policy or bucket-policy conditions. The fact that “it works for me” is a lead to compare contexts, not evidence that the worker role is defective.

Change the mismatched key before changing a policy. If the key is wrong, fix the reference path and rerun the same narrow request. If the key is correct and independently present, move on to authorization evidence. Changing the key and several policies in one attempt prevents you from learning which condition mattered.

When should you investigate an actual denial?#

Once the target and identity are established, inspect the denial information and policies relevant to that request. AWS documents explicit and implicit denial cases across several policy types. An enhanced error message can report only one reason even when more than one applies, so resolving its named issue may expose another. ^[3]

For example, an identity allow does not override an applicable explicit deny. Bucket or access-point policy, session restrictions, permissions boundaries, organization controls and network endpoint conditions can change the effective result. The exact evaluation depends on the principal and resource relationship; avoid reducing cross-account access to a single universal “either policy is enough” rule. ^[3][4]

Use the denial’s named policy type when available, then verify action, resource ARN and conditions. Distinguish bucket resources from object resources. s3:GetObject permission on a bucket ARN without its object path is not the same as permission on objects under a prefix. Also distinguish a runtime application requirement from a diagnostic permission used by an administrator.

Do not expand the worker to s3:* to see what happens. A broad temporary policy can hide the original cause, expose data and become permanent when the incident ends. Make any required policy correction explainable as a specific action on a specific resource under the required conditions, then rerun the unchanged request.

CloudTrail or access logs can help when they are already configured with the right coverage. Their absence is not proof that the request never happened. Verify whether the relevant data events are collected, where they are delivered and what delay is expected before relying on them. Retain the request identifier for the owner or support investigation.

Why can HEAD succeed while the download still fails?#

Because the later request has a different contract. A metadata-only HEAD does not download the object. Encrypted objects, checksum retrieval options, archive storage and version selection can introduce additional requirements. Test the application’s actual retrieval operation after fixing the metadata path. ^[1][6]

For SSE-KMS, examine the exact request options and the relevant key authorization. The HeadObject reference documents additional KMS permissions for checksum retrieval; do not assume every ordinary HEAD needs the same options as an upload. Sending upload-style encryption headers on a metadata request can also produce a different error. ^[1]

A successful HEAD for an archived object does not mean the body is immediately retrievable. The GetObject reference describes storage classes that require restoring a copy first. That should become a product-visible state or a separate retrieval workflow, not an endless loop of permission changes. ^[6]

Do not perform a large download merely to establish that the metadata problem changed. Use a synthetic disposable object for end-to-end authorization tests, or an appropriately scoped request agreed with the owner. The verification goal is the intended operation with controlled cost and exposure. An investigation should not accidentally turn into bulk data access.

For this scenario, record two outcomes separately: the application now uses the authoritative key, and the worker can perform the required metadata/body operations for its own export. The first is a reference correctness check. The second is an access and storage-state check. Neither should be inferred from the other.

What should the application return while the cause is unknown?#

Do not translate every S3 403 into a user-facing “file exists but access is forbidden” message. The storage service intentionally leaves existence ambiguous for some callers, and the application may not have enough evidence to say more. Return a stable product error and retain a diagnostic identifier for investigation. ^[1]

Where the product owns a trusted export record, it can express the record’s known state: preparation incomplete, ready, expired or needs investigation. Keep that state model separate from a single storage response code. A missing object behind a ready record is a consistency problem the application should reconcile.

Add tests for the path builder and error classifier using fake SDK responses. Include an existing authorized object, an absent key without list visibility, an explicit denial, a mismatched key and a stale manifest. The classifier should preserve uncertainty rather than returning exists=True for 403. These tests establish your application’s interpretation; they do not validate a real AWS policy. ^[1]

The release record should show the exact repaired reference, unchanged runtime privilege where applicable, the controlled comparison request and the required end-to-end operation. If a policy did need changing, keep the specific reason and scope beside that record. A transition from 403 to a more informative error is useful diagnostic progress, not automatic proof that the export works. ^[1]

If the worker obtains AWS credentials through CI, the GitHub Actions OIDC subject investigation helps establish which identity the job actually assumed.

Can S3 return 403 for a nonexistent key?

Yes. The HeadObject documentation says the missing-object response depends on s3:ListBucket: callers without that permission can receive 403 instead of 404. That response alone does not establish whether the key exists. ^[1]

Does every HeadObject caller need ListBucket?

No. Do not turn a permission that affects missing-key visibility into a universal runtime requirement. Evaluate the metadata operation and the product’s actual listing needs separately. Preserve the narrow role if the application only needs known object references. ^[1]

Why does my administrator profile work while the worker fails?

The credentials and request context differ. Compare identity, policies, endpoint/network conditions, bucket/key/version and request options. A successful administrator request can help the owner verify existence, but it does not reproduce the worker’s authorization decision. ^[3][5]

Does a successful HEAD prove the object body can be downloaded?

No. HEAD returns metadata. Test the actual retrieval requirements, including encryption authorization and archive state, through a controlled request. A metadata success is useful evidence for the HEAD operation, not a substitute for the body-retrieval contract. ^[1][6]