Skip to content

AWS Updated 2026-07-04

S3 Inventory and Storage Lens: seeing what is actually in your buckets

How to use S3 Inventory and S3 Storage Lens to see what is actually in your buckets — object-level reports queryable with Athena, fleet-wide cost metrics, and the setup for both.

You cannot manage S3 spend you cannot see, and aws s3 ls stops being a visibility tool somewhere around a million objects. The two built-in answers split the job: Storage Lens gives fleet-wide metrics across every bucket in the account or organization — including the money sinks object listings hide, like noncurrent versions and incomplete multipart uploads — while S3 Inventory produces a scheduled object-by-object report you query with Athena when you need to know exactly which keys, prefixes, and storage classes the bytes live in. Storage Lens tells you where to look; Inventory tells you what is there.

What’s happening

Storage Lens aggregates daily metrics without touching your objects. The default dashboard is free, covers every bucket, retains 14 days, and already includes the cost-relevant metrics most teams never look at: total bytes by bucket, object count, average object size, noncurrent version bytes, delete marker counts, and incomplete multipart upload bytes. Advanced metrics are a paid upgrade that adds prefix-level aggregation, activity metrics (requests, downloads), 15 months of retention, and CloudWatch publishing. For most accounts, the free tier answers the first, most valuable question: which buckets hold the bytes, and how much of that is invisible overhead.

S3 Inventory works differently: on a daily or weekly schedule, S3 writes a manifest plus data files (CSV, ORC, or Parquet) into a destination bucket you choose, listing every object — or every object under a filter — with optional fields like size, storage class, last-modified, encryption status, replication status, and ETag. It is eventually consistent (the report reflects a point-in-time-ish snapshot, not the live bucket), the first report can take up to 48 hours to arrive, and the report itself costs a small per-object fee plus the storage it lands on. Its two killer uses: Athena queries over millions of objects without paying LIST request costs, and ready-made manifests for S3 Batch Operations jobs like mass Glacier restores. Field-by-field details are in the S3 documentation.

What to check first

  • Whether Storage Lens is already on. The default dashboard exists automatically — most teams have simply never opened it.
  • Which question you are actually asking. “Which bucket costs the most” is Storage Lens. “Which 2019 prefixes are still in Standard” is Inventory plus Athena.
  • A destination bucket for Inventory reports. Same region as the source, with a bucket policy allowing S3 to deliver.
  • Whether versioning is in play. If yes, configure Inventory to include all versions — current-only reports hide the exact bloat you are hunting.

How to verify

Open S3 > Storage Lens > Dashboards > default-account-dashboard and check three numbers per top bucket: total bytes, noncurrent version bytes, and incomplete multipart upload bytes. Those last two are pure waste candidates. From the CLI, list what Inventory configurations already exist on a bucket:

aws s3api list-bucket-inventory-configurations --bucket my-bucket

Set one up (Parquet, weekly, all versions, the useful fields):

aws s3api put-bucket-inventory-configuration \
  --bucket my-bucket \
  --id weekly-audit \
  --inventory-configuration '{
    "Id": "weekly-audit",
    "IsEnabled": true,
    "IncludedObjectVersions": "All",
    "Schedule": {"Frequency": "Weekly"},
    "Destination": {"S3BucketDestination": {
      "Bucket": "arn:aws:s3:::my-inventory-reports",
      "Format": "Parquet"
    }},
    "OptionalFields": ["Size","LastModifiedDate","StorageClass","IsLatest","EncryptionStatus"]
  }'

The destination bucket needs a policy allowing the s3.amazonaws.com service principal to PutObject for reports; the console offers to add it when you configure Inventory there instead.

The runbook

  1. Start with the free Storage Lens dashboard. Rank buckets by size, then scan noncurrent version bytes and incomplete MPU bytes. This ten-minute pass usually finds the first cleanup targets — often fixable the same day with lifecycle rules.

  2. Turn on Inventory for the buckets that matter. The big ones and the audit-sensitive ones. Parquet format — Athena scans it far cheaper than CSV.

  3. Create the Athena table over the reports. The console’s Inventory page links a ready-made template. Then ask the questions listings cannot answer:

    SELECT storage_class,
       count(*) AS objects,
       round(sum(size)/1073741824.0, 1) AS gib
FROM my_bucket_inventory
WHERE dt = '2026-06-29-01-00'
GROUP BY storage_class
ORDER BY gib DESC;

    Swap the GROUP BY for a prefix expression (split_part(key, '/', 1)) to find which top-level prefixes hold the bytes.

  4. Act on what you find, using the report as the manifest. Inventory output feeds S3 Batch Operations directly — tag, copy, restore, or change class for exactly the keys the query surfaced.

  5. Consider advanced Storage Lens metrics only after the free tier runs dry. Prefix-level aggregation and activity metrics are genuinely useful on large estates; on a five-bucket account they are a fee for confirming what you already know.

When this is the wrong move

For a bucket with a few thousand objects, aws s3 ls --summarize and the console answer everything faster than setting up reports. If the question is “who accessed what”, that is CloudTrail data events or S3 server access logs — neither Inventory nor Storage Lens records access by identity. And if you need a guaranteed-complete list at one exact instant (a legal snapshot), Inventory’s eventual consistency is the wrong instrument.

Common mistakes

  • Configuring Inventory with current versions only on a versioned bucket, then concluding versioning overhead does not exist.
  • Waiting on the first report without knowing it can take up to 48 hours, and re-creating the configuration repeatedly in the meantime.
  • Delivering reports into the same bucket being inventoried without a prefix filter — each report then inventories the previous reports.
  • Reading Storage Lens totals as billing-exact; it is a metrics system for finding problems, and the invoice remains the source of truth.

Keeping it under control

  • Make the Storage Lens dashboard a monthly fifteen-minute ritual: top buckets, growth trend, waste metrics.
  • Keep Inventory on for any bucket above the size where listing is impractical, and lifecycle-expire old reports in the destination bucket.
  • Save the three or four Athena queries the team actually uses next to the infrastructure code.
  • Feed cleanup actions from Inventory manifests through Batch Operations rather than ad-hoc scripts, so there is a job report when someone asks what changed.

Visibility is the unglamorous half of S3 cost work, and it is the half that makes every other fix in this series a targeted action instead of a guess. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.