Skip to content

Designing a backup retention policy — daily, weekly, monthly, yearly

A practical guide to backup retention design — the purpose of each tier, the cost vs recovery-time trade-offs, and the Canadian compliance overlays that shape the schedule.

Most teams set their backup retention by accident. Someone installed the backup tool, the wizard suggested “keep 30 days of daily backups,” they clicked next, and the policy stuck. Two years later the bill for backup storage is ten times the bill for production storage, the auditor is asking for a backup from sixteen months ago that no longer exists, and the recovery team can no longer answer the question “if something broke 47 days ago, can we still recover?” The retention policy was never designed; it was inherited.

Retention is a design problem, not a default. Every backup costs money to keep, costs more money to retrieve, and answers a specific recovery scenario. A well-designed policy spells out which scenarios it covers, which tier covers each, and what is acceptable to lose. This article walks through the four-tier grandfather-father-son model, the per-tier purposes, the Canadian compliance overlays, the storage shape and key rotation choices, and the failure modes that look fine for years and then suddenly aren’t.

The two questions retention policy actually answers

Most retention discussions get lost in unit conversions — “we want 30 days but the compliance policy says 7 years” — without first agreeing on what retention is for. There are only two questions, and they are pointed at different audiences.

“If something broke 47 days ago, can I recover?” This is the engineering question. It assumes a corruption or a deletion or a ransomware event whose blast radius is bounded by time, and asks whether the backup tier still holds a clean copy from before the event. The answer depends entirely on how long the gap between event and detection is — and the gap is much longer than most teams assume. The median ransomware detection window is around 200 days; the median silent-corruption detection window for a slowly-mutating dataset is longer.

“Can I prove this to my auditor, regulator, or customer?” This is the compliance question. PIPEDA, Law 25 in Quebec, PCI DSS, HIPAA, the SEC’s Rule 17a-4, the CRA’s seven-year records-retention guidance — each has a horizon, each requires evidence of retention, and each requires evidence that the retained data is restorable. The compliance horizon is almost always longer than the engineering horizon, and the cost shape is very different — cold storage with rare retrieval rather than hot storage with daily restore tests.

A retention policy that answers only the engineering question fails an audit. A policy that answers only the compliance question cannot recover from a Wednesday accident. A useful policy tiers the answers — daily and weekly for engineering, monthly and yearly for compliance — and is explicit about which tier covers which event.

The grandfather-father-son rotation — what it is, when it fits

Grandfather-father-son (GFS) is the rotation that became the default for filesystem backups in the 1990s and remains the default for database backups today. Three rolling buckets:

  • Son — daily backups, kept for a short window. Each day a new son is added; an old son rolls off.
  • Father — one weekly backup, kept for a longer window. One son per week is promoted to a father; one father rolls off after enough weeks.
  • Grandfather — one monthly backup, kept for the longest window. One father per month is promoted to a grandfather; one grandfather rolls off after enough months.

Modern policies extend GFS with a fourth tier — great-grandfather, or just “yearly” — for compliance horizons that exceed three to five years. The promotion logic is the same: one monthly per year becomes a yearly; yearly rolls off after the regulatory horizon, which can be twenty-five years for some Canadian financial workloads.

GFS fits any workload where the backup is a logical snapshot at a point in time and where storage cost grows linearly with retained copies. It does not fit continuous-data-protection schemes that retain every change for the entire window — those need different math. For database backups, file backups, VM snapshots, and most cloud-native backup services, GFS is the right starting model.

Per-tier purpose

Each tier has a distinct purpose, a distinct recovery scenario, and a distinct cost shape. Designing them as one undifferentiated bucket — “keep 90 days of everything” — costs more and recovers less than designing each tier for what it is for.

Daily backups — for the accident yesterday

Daily backups cover the events with the shortest detection windows: an engineer dropped the wrong table, a user deleted the wrong file, a deploy corrupted the most recent rows in a hot table. They also cover the early stages of a ransomware event before the dwell time has consumed the older copies. Typical retention is 7 to 30 days; the lower end fits small low-mutation workloads with strong upstream change-control, the higher end fits busy multi-user systems where the corruption window is often measured in weeks.

Daily backups live on hot storage. They are the tier most often restored from, the tier verified most frequently (Level 1 and Level 2 every night, Level 3 weekly), and the tier least sensitive to per-GB storage cost. Optimise for restore latency rather than archive density.

Weekly backups — for the “we noticed last week” event

Weekly backups cover the awkward middle: corruption that began a few weeks ago but was masked by application caching, a slow data poisoning campaign, an integration with a partner that started writing bad records ten days ago and was caught yesterday. They also cover the gap when the daily tier is consumed by a single very long incident.

Typical retention is 4 to 12 weeks. Storage class is usually warm (Standard-IA on AWS, equivalent on other providers); they are restored from occasionally but not frequently enough to justify the cost of hot storage. Cadence is one per week — pick the same day each week, usually Sunday or Monday, and promote the daily backup taken on that day.

Monthly backups — for compliance and slow corruption

Monthly backups serve two masters: the engineering case of corruption with a months-long detection window, and the early compliance horizon where most regulators expect at least monthly archival. Typical retention is 12 to 36 months.

By the time a workload hits its monthly tier, it has aged out of the hot recovery scenarios; the storage class shifts to cold (Glacier on AWS, equivalent on other providers). Restore latency moves from minutes to hours; this is acceptable because the recovery scenarios that reach into the monthly tier are not 03:00 fires but planned recoveries with stakeholder coordination. The cost trade-off is dramatic — per-GB storage in Glacier is a small fraction of Standard, but per-restore retrieval cost is a multiple of Standard’s egress.

Yearly backups — for the regulator, the auditor, and the lawsuit

Yearly backups exist because the law says so. Canadian PIPEDA and provincial equivalents have evolved retention expectations; Quebec’s Law 25 strengthens this. The Canada Revenue Agency expects six years of corporate financial records, often interpreted as seven to align with the calendar year and fiscal close cycles. PCI DSS requires at least one year of audit-log retention with three months immediately available. Provincial securities commissions and SEC Rule 17a-4 demand seven years for some financial workloads; certain healthcare data has retention horizons of twenty-five years or longer.

Typical retention for the yearly tier is 7 to 25 years depending on jurisdiction and workload. Storage class is the coldest tier available — Glacier Deep Archive on AWS, Archive on Azure, equivalents on other providers. Restore latency is measured in 12 to 48 hours. The cost shape is dominated by retrieval rather than storage; the data sits for years untouched, then a single legal-hold or audit triggers a one-off retrieval whose egress cost is the dominant line item in the year’s backup spend.

The Canadian context — PIPEDA, Law 25, and overlays

Stack Harbor’s customers operate primarily under Canadian privacy and records-retention regimes. The relevant horizons that shape retention design:

  • PIPEDA — the federal Personal Information Protection and Electronic Documents Act. Personal information must be retained “only as long as necessary for the purpose for which it was collected”; the OPC’s interpretive guidance pushes toward conservative retention with disposal evidence. Practically, organizations retain 7 years to cover overlap with tax law and contract limitation periods.
  • Quebec Law 25 (formerly Bill 64) — provincial private-sector privacy law, stricter than PIPEDA in several respects. It tightens disposal obligations and requires evidence of secure destruction, not just removal from the active system. Retention policies covering Quebec residents need disposal procedures that document the secure end-of-life of each yearly archive when it ages out.
  • CRA records — six years from the end of the fiscal year for most business records; often retained as seven calendar years.
  • PCI DSS for any payment workloads — at least one year of audit logs, with three months immediately available, plus retention of cardholder data subject to disposal rules.
  • PIPA Alberta and PIPA BC — provincial overlays that may apply depending on where customers reside.
  • Data residency for Quebec consumer data — Law 25 does not impose absolute residency but raises evidentiary requirements for cross-border processing; many Quebec workloads keep at least the yearly tier in Canadian regions.

Healthcare and finance overlays vary by province and by federally-regulated status; retention horizons of 10, 25, or even 50 years apply in narrow cases. The right approach is to write the policy with the longest applicable horizon as the yearly retention and to file the policy with a citation to each governing rule. Auditors accept a documented horizon with a citation; they do not accept “we keep yearly forever.”

Storage shape — hot vs warm vs cold

The cost-recovery trade-off across storage classes is the lever that makes long retention financially sustainable. The three classes that matter for backup retention:

  • Hot (S3 Standard, equivalents). Per-GB storage cost is the highest; per-request and per-retrieval costs are zero or negligible. Restore latency is milliseconds. Use for the daily tier and for the tier currently being verified.
  • Warm (S3 Standard-IA, S3 Intelligent-Tiering, equivalents). Per-GB storage cost is a fraction of hot, in exchange for a per-GB retrieval fee and a minimum storage duration. Restore latency is still effectively instant. Use for the weekly tier and sometimes the first months of the monthly tier.
  • Cold (Glacier, Glacier Deep Archive, equivalents). Per-GB storage cost approaches the lowest commercially available; retrieval fees and retrieval latencies are substantial. Restore latency ranges from minutes (Glacier Instant Retrieval) to hours (Standard) to 12-48 hours (Deep Archive). Use for the monthly tier (mid-range Glacier) and the yearly tier (Deep Archive).

S3 lifecycle policies, or the equivalent on other providers, automate the transition. A backup lands in Standard, transitions to Standard-IA after 30 days, to Glacier after 90 days, to Deep Archive after 365 days, and is deleted at the end of the retention horizon. The lifecycle policy is the policy — if it is correct, the bill follows from it.

A frequent mistake is to default everything to the coldest available class. This optimises storage cost and demolishes restore time, which is fine until a Friday afternoon Sev-1 needs the weekly backup and the 12-hour retrieval window from Deep Archive makes the RTO commitment impossible.

Encryption and key rotation — the multi-decade keychain

A backup that you cannot decrypt is worse than no backup; it gives the auditor a false comfort. The yearly tier is where this matters most, because the encryption key that was used to write the archive must still be available, derivable, or restorable seven to twenty-five years later.

Three practices that age well over multi-decade horizons:

  • Use an envelope encryption scheme. Each archive is encrypted with a fresh data-encryption key (DEK) which is itself encrypted by a long-lived key-encryption key (KEK). The KEK is what needs to survive; rotating the KEK does not invalidate older archives.
  • Manage KEKs in a service with documented retention. AWS KMS, Azure Key Vault, GCP KMS each support customer-managed keys with policy-controlled retention. Deleting a key revokes access; pending-deletion windows give time to recover. The KEK rotation history is auditable.
  • Verify decryption of the oldest archive, not just the newest. The Level 1 verification in the previous article applies here. Each verification cycle, fetch the oldest archive in the retention window, decrypt the header, and assert the integrity check passes. A KEK that was deleted three years ago does not surface until someone tries to use it.

A separate operational rule: never store the KEK or its passphrase in the same blast radius as the data. A ransomware event that encrypts the backup store and the credential store together leaves no key recovery path.

Calculating cost shape

Backup retention cost has three components — storage, requests, and retrieval. Across a multi-tier policy the proportions shift dramatically.

  • Daily tier. Storage cost dominates; retrieval cost is negligible because restores are frequent but each restore costs little on hot storage.
  • Monthly tier in Glacier. Storage cost is modest; retrieval cost is moderate because there are a handful of audit-driven restores per year.
  • Yearly tier in Deep Archive. Storage cost is small; retrieval cost is concentrated in rare legal-hold events whose egress dominates the year’s backup spend.

The qualitative pattern: the colder the tier, the lower the storage burn rate but the higher the per-event retrieval risk. A policy that ignores the retrieval-cost shape is fine for years and is then surprised by a single audit cycle whose Glacier egress costs more than a year of hot-tier storage.

Mistakes that look fine until they don’t

A few patterns recur in retention policies that look healthy on the dashboard and fail at a moment of need.

  • “We have S3 lifecycle to Glacier” but no test restore from Glacier. The lifecycle policy moves data to Deep Archive; nobody verifies retrieval until the audit cycle hits and the 48-hour standard retrieval blows past the auditor’s deadline. Test retrieval from the coldest tier at least annually, on a known archive whose checksum is recorded.
  • Retention shorter than the gap-detection window. Corruption began on Day 1, was noticed on Day 35, and the only clean backup is Day 0 — which expired on Day 30. The fix is to choose the daily tier length based on the actual detection lag for the workload, not the default that came with the tool.
  • Compliance tier stored in the same account or zone as the day tier. A ransomware actor with privileged access encrypts the day, week, and yearly tiers in one motion because they all live in the same account. The compliance tier belongs in a separate account, with separate credentials, immutable bucket-lock or object-lock, and ideally air-gapped from the operational identity domain.
  • Yearly retention without a disposal procedure. Law 25 expects evidence of secure destruction at end-of-life. A policy that “keeps yearly forever” is not actually a policy; it is the absence of one.
  • Verification only on the newest backup. Reduces verification cost but means the oldest backup — the one most likely to surface a decryption or format-drift problem — is never tested until the day it is needed. Verification cadence must touch the oldest archive in the window, at least once per year.
  • No documented owner. The retention policy is in a wiki page; the wiki page was last edited four years ago; nobody knows whether the current settings still match the intent. Retention policies need a documented owner and a quarterly review, even if the review is “no change.”

When Stack Harbor designs and runs this for you

We design retention policies for managed-operations customers as part of the onboarding for managed operations — the policy is reviewed quarterly with the account manager, the lifecycle rules are committed to infrastructure-as-code, and the verification ladder described in the database backup verification article runs against each tier on the appropriate cadence.

For multi-region workloads, the same retention policy applies in both regions; the yearly tier in the recovery region is treated as the audit-of-last-resort copy and is exercised quarterly as part of the multi-region environments DR drill. For cloud-compute customers we offer retention design as a one-off advisory engagement; the policy and the lifecycle IaC are handed over with documentation suitable for the client’s own audit cycle.

If you already have a retention policy and want a second opinion on whether it actually answers the two questions at the start of this article, we offer a retention audit — three weeks, one deliverable, a tier-by-tier review with a fix list and a cost-shape model.

See also