Skip to content

AWS Updated 2026-03-17

AWS Backup with cross-region and cross-account copy — the restore-test that proves it actually works

AWS Backup centralizes backup across EBS, RDS, EFS, DynamoDB, S3. Backup plans, vaults with cross-region and cross-account copy, KMS, and the restore-test loop that turns a green backup into proven recoverable data.

A disaster recovery exercise on a customer environment in 2025 surfaces the gap nobody had noticed. AWS Backup is running, the backup plan covers RDS, EBS, EFS, and DynamoDB, the CloudWatch metric NumberOfBackupJobsCompleted shows green for 180 consecutive days. Then the team tries to restore the production database from a 30-day-old recovery point into a sandbox account for a compliance audit, and the restore fails — the IAM role that AWS Backup is configured to assume in the sandbox account does not have the right trust policy, the KMS key in the source account does not grant decrypt to the sandbox role, and the actual recovery from the cross-account vault has never been exercised. The backup was running. The restore was not proven. The setup below is the one that proves it.

What’s happening

AWS Backup is a managed orchestration layer that takes snapshots across multiple AWS resources on a schedule and stores them in vaults. The supported resources expanded steadily — EBS volumes, RDS clusters, EFS file systems, DynamoDB tables, FSx, S3, Aurora, Storage Gateway, VMware, Neptune, DocumentDB, Redshift, SAP HANA, Timestream. For most workloads, AWS Backup replaces the per-service snapshot mechanism (EBS lifecycle, RDS automated backups) with one centralized policy and one set of vaults.

The architecture has three layers:

  • Backup plan — schedule, retention, lifecycle (cold storage transition), copy actions.
  • Backup vault — destination for recovery points, encrypted with a KMS CMK, with optional Vault Lock for immutability.
  • Resource assignment — selects which resources the plan applies to, by tag or by ARN list.

The cross-region/cross-account piece is the disaster-recovery layer. A backup plan can copy each recovery point to a second vault in another region, another account, or both. The destination vault has its own KMS CMK, its own retention policy, its own Vault Lock state. A regional disaster in us-east-1 does not take the cross-region copy with it; a compromise of the production account does not take the cross-account copy with it (the destination vault is in a separate AWS account with separate credentials).

The “backup runs but restore was never tested” failure mode is endemic. The metric NumberOfBackupJobsCompleted says nothing about whether the resulting recovery point is restorable. The proof comes from actually running a restore, on a cadence, with the destination being a real recovery (not a no-op API call that returns success without doing the work).

How to verify

The state of the current setup:

aws backup list-backup-plans \
  --query 'BackupPlansList[].[BackupPlanName,BackupPlanId,LastExecutionDate]' \
  --output table

aws backup list-backup-vaults \
  --query 'BackupVaultList[].[BackupVaultName,EncryptionKeyArn,NumberOfRecoveryPoints,Locked]' \
  --output table

For each plan, get the full details — the schedule, the lifecycle, the copy actions:

aws backup get-backup-plan \
  --backup-plan-id <id> \
  --query 'BackupPlan.Rules[].[RuleName,ScheduleExpression,Lifecycle,CopyActions]' \
  --output yaml

A healthy plan has at least one rule with a CopyAction pointing to a vault ARN in a different region (and ideally a different account). If CopyActions is empty, the backup has no cross-region copy.

For the restore-test verification:

aws backup list-restore-jobs \
  --by-created-after $(date -u -v-90d +%Y-%m-%dT%H:%M:%S) \
  --query 'RestoreJobs[].[CreationDate,Status,RecoveryPointArn,ResourceType]' \
  --output table

The empty case — no restore jobs in the last 90 days — means the restore has not been exercised. The first restore at incident-time is going to discover whatever configuration gap the team did not test in advance.

The fix

The full setup, deployable as one CloudFormation stack in each of three accounts (workload, backup, DR):

  1. Backup account vault setup. A dedicated AWS account that owns the centralized backup vault. The KMS CMK is created in the backup account; the workload accounts get explicit grant on the key for kms:Encrypt only (so they can write encrypted backups but cannot decrypt the resulting recovery points without invoking AWS Backup):

    BackupVault:
  Type: AWS::Backup::BackupVault
  Properties:
    BackupVaultName: central-prod-backup
    EncryptionKeyArn: !GetAtt BackupKmsKey.Arn
    AccessPolicy: !Ref VaultAccessPolicy

VaultLockConfiguration:
  Type: AWS::Backup::BackupVault
  Properties:
    BackupVaultName: central-prod-backup
    LockConfiguration:
      MinRetentionDays: 7
      MaxRetentionDays: 2555
      ChangeableForDays: 3

    The ChangeableForDays: 3 window is the cooling-off period: the vault lock is reversible during the first three days, then it locks in compliance mode and cannot be deleted. Production vaults should reach compliance mode within a week of going live.

  2. DR account vault setup. Identical pattern in a separate AWS account in a different region. The Backup plan’s copy action targets this vault.

  3. Workload account: backup plan, IAM role, and resource assignment:

    BackupRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: "2012-10-17"
      Statement:
        - Effect: Allow
          Principal: { Service: backup.amazonaws.com }
          Action: sts:AssumeRole
    ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup
      - arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForRestores

ProdBackupPlan:
  Type: AWS::Backup::BackupPlan
  Properties:
    BackupPlan:
      BackupPlanName: prod-daily
      BackupPlanRule:
        - RuleName: daily
          TargetBackupVault: prod-local-backup
          ScheduleExpression: "cron(0 5 ? * * *)"
          Lifecycle:
            DeleteAfterDays: 35
          CopyActions:
            - DestinationBackupVaultArn: arn:aws:backup:us-east-1:222222222222:backup-vault:central-prod-backup
              Lifecycle:
                DeleteAfterDays: 365
            - DestinationBackupVaultArn: arn:aws:backup:eu-west-1:333333333333:backup-vault:dr-prod-backup
              Lifecycle:
                DeleteAfterDays: 365

ResourceAssignment:
  Type: AWS::Backup::BackupSelection
  Properties:
    BackupPlanId: !Ref ProdBackupPlan
    BackupSelection:
      SelectionName: prod-tagged
      IamRoleArn: !GetAtt BackupRole.Arn
      ListOfTags:
        - ConditionType: STRINGEQUALS
          ConditionKey: backup-tier
          ConditionValue: prod

    The two copy actions in the plan rule produce three copies per backup: local (35-day retention), central in another account (365-day retention), DR in another region (365-day retention).

  4. Restore-test pipeline. A scheduled job (EventBridge cron + Lambda or a CodeBuild project) that performs a restore from the DR vault every two weeks. The job:

    • Picks the most recent recovery point of one chosen resource (a non-critical RDS sandbox database is a good target).
    • Initiates the restore into a sandbox account.
    • Polls the restore status until completion.
    • Verifies the restored resource exists, queries it briefly to confirm data is readable.
    • Deletes the restored resource.
    • Reports success/failure to the on-call channel.

    The chosen resource matters: testing the restore on the production database itself would be disruptive, so the test target is a sandbox copy that has the same schema and roughly the same data volume. The point is to exercise the IAM trust path, the KMS grant, the network configuration, and the resource-creation path — not to validate the production data itself.

  5. CloudWatch alarm on backup job failure. The alarm from the baseline article in this series covers NumberOfBackupJobsFailed > 0 over an hour. The restore-test pipeline contributes its own metric — RestoreTestFailureCount published to CloudWatch — which is the second alarm.

Three rules across every backup-implementation engagement:

Cross-region and cross-account, both. Cross-region defends against regional outages and regional disasters. Cross-account defends against credential compromise. The two together are the floor — either one alone has a real failure mode.

Vault Lock in compliance mode. The lock prevents ransomware operators (or a compromised admin) from deleting recovery points. Production vaults should be in compliance mode within a week of going live. The cooling-off window is short because the lock is the security control that justifies the backup.

Restore-test on a fortnightly cadence. Backup metrics say the backup ran. Restore tests say the recovery point is recoverable. The first restore at an incident is too late to discover the IAM trust is wrong.

The full deployment on a fresh AWS Organization is a focused three days — most of it is the IAM trust matrix between the workload, backup, and DR accounts. The ongoing cost is two hours every quarter for the restore-test review. The exercise that found the cross-account restore gap in the opening story would have caught it within the first two weeks of running the restore-test pipeline.

Stack Harbor runs IAM cleanup, CloudWatch baseline, AWS Backup, and the cost-control rhythm as part of Managed AWS and ongoing Managed Cloud Ops. If the backup metric is green but nobody has restored from it in six months, start with an infrastructure consult.