Skip to main content
Graduate Track

Locking Down Production RDS Backups with AWS Backup Vault Lock

A hands-on runbook for locking production RDS backups with AWS Backup Vault Lock and cross-region copy, covering rollback limits and audit evidence.

Locking Down Production RDS Backups with AWS Backup Vault Lock
Priya NairPriya Nair12 min readProduction Ready75 min

In this lesson

Share

Before you begin

  • AWS IAM permissions and role management
  • AWS CLI configuration and usage
  • RDS instance operations basics

Track this tutorial

Choose your current status and tick each safety check as you complete it. Sign in to sync progress between devices.

Current status

Tutorial stages

0 of 4 stages complete

Before you apply the change

Confirm these production-safety controls during the tutorial.

0 of 6 safety checks completed

#Operational requirement

You are going to get handed a ticket that reads something like: “harden backups

against ransomware, DR audit next quarter.” Take it seriously. A previous incident that triggered this exact requirement involved a compromised IAM access key. The attacker deleted the production RDS instance, then walked through the AWS Backup console and deleted every recovery point in the vault, because the stolen role happened to carry backup:DeleteRecoveryPoint and backup:DeleteBackupVault. There was nothing left to restore from. That single fact is the reason this procedure exists.

AWS Backup Vault Lock makes recovery points immutable, and in compliance mode it makes them deletion-proof against every principal in the account, including root. Combine it with cross-region copy and you also satisfy the availability requirement: a regional outage, or an attacker who only has access to one region, cannot destroy your only copy of production data. This is a realistic task for a newly hired administrator responsible for RDS resilience. It is also unforgiving. Get the lock mode wrong and there is no undo button.

Do not point this procedure at production on your first attempt. Build it against a throwaway vault first.

#Prerequisites and required permissions

You need an existing production RDS instance, a change management process that will genuinely approve an irreversible action, and the following permissions attached to your working role, ideally through a scoped permission set rather than administrator access: backup:CreateBackupVault, backup:CreateBackupPlan, backup:CreateBackupSelection, backup:StartBackupJob, backup:PutBackupVaultLockConfiguration, backup:DeleteBackupVaultLockConfiguration, kms:CreateKey, kms:CreateGrant, and iam:PassRole scoped to the AWS Backup service role.

You also need a second AWS region already reachable from the account, with no Organizations SCP blocking it, MFA enforced on the identity performing the lock operation, and a Change Advisory Board reference number written into the ticket before you start. Vault Lock in compliance mode is one of the few AWS actions where “we’ll fix it later” simply does not exist as an option. The approval trail matters more here than on almost any other change you will make this year.

#Step 1: Create backup vaults with dedicated KMS keys in both regions

Create a customer-managed KMS key per region rather than accepting the AWS managed backup key. You need explicit key policy control later for the cross-region copy step.

1aws kms create-key --region us-east-1 --description "prod-rds-backup-primary"
2aws backup create-backup-vault 
3  --backup-vault-name prod-rds-vault 
4  --encryption-key-arn arn:aws:kms:us-east-1:111122223333:key/PRIMARY-KEY-ID 
5  --region us-east-1
6
7aws kms create-key --region us-west-2 --description "prod-rds-backup-secondary"
8aws backup create-backup-vault 
9  --backup-vault-name prod-rds-vault-dr 
10  --encryption-key-arn arn:aws:kms:us-west-2:111122223333:key/SECONDARY-KEY-ID 
11  --region us-west-2

Attach resources by tag, not by explicit ARN, so future RDS instances inherit protection automatically without a second change ticket:

1aws backup create-backup-selection 
2  --backup-plan-id <plan-id-from-previous-output> 
3  --backup-selection '{"SelectionName":"prod-rds-tagged","IamRoleArn":"arn:aws:iam::111122223333:role/AWSBackupDefaultServiceRole","ListOfTags":[{"ConditionType":"STRINGEQUALS","ConditionKey":"backup:tier","ConditionValue":"production-rds"}]}' 
4  --region us-east-1

Expect a SelectionId returned with no error. Record the plan ID and selection ID in the ticket. Tag the RDS instance with backup:tier=production-rds before moving on, and give it at least fifteen minutes before the next evaluation window. Miss that window and the first job silently skips the resource, with no error surfaced anywhere.

#Step 3: Validate the service role and run a manual backup job

Confirm the AWS Backup service role carries both managed policies, not just one:

aws iam list-attached-role-policies --role-name AWSBackupDefaultServiceRole

You need AWSBackupServiceRolePolicyForBackup for the backup jobs themselves and AWSBackupServiceRolePolicyForRestores for the restore test in Step 4. Force a manual run instead of waiting on the cron schedule:

1aws backup start-backup-job 
2  --backup-vault-name prod-rds-vault 
3  --resource-arn arn:aws:rds:us-east-1:111122223333:db:prod-orders-db 
4  --iam-role-arn arn:aws:iam::111122223333:role/AWSBackupDefaultServiceRole 
5  --region us-east-1

Run the command, then poll aws backup describe-backup-job --backup-job-id <id> until State reads COMPLETED. Separately confirm a matching copy job with aws backup list-copy-jobs --by-state COMPLETED --region us-east-1, showing a DestinationRecoveryPointArn in us-west-2. If the copy job sits in RUNNING past the completion window, or fails on an access-denied condition, stop. Fix the destination KMS key policy before you go anywhere near Vault Lock.

#Step 4: Apply Vault Lock, governance mode first

ModeCan be removed?When to use it
Governance (within grace period)Yes, by an authorised principalTesting, initial rollout, staging environments
Compliance (after grace period expires)No, not by anyone, including rootFinal production state once tested and CAB-approved

Vault Lock uses a single configuration call. During the grace period you set with --changeable-for-days, it behaves like governance mode and an authorised principal can still remove it. Once that grace period elapses, the lock converts automatically and permanently into compliance mode. There is no warning prompt for this transition.

1aws backup put-backup-vault-lock-configuration 
2  --backup-vault-name prod-rds-vault 
3  --min-retention-days 35 
4  --max-retention-days 100 
5  --changeable-for-days 3 
6  --region us-east-1

Expect no error, and a locked configuration block in the next describe-backup-vault call. Capture the full describe output with a timestamp and the calendar date the grace period expires, and put both in the ticket. Do not extend the grace period past your CAB-approved test window. If restore testing is not finished before the grace period ends, delete and reapply the lock rather than letting it convert unreviewed.

#Verification

Before closing the change ticket, run all of the following and attach the outputs:

AWS Backup Vault Lock
1aws backup describe-backup-vault --backup-vault-name prod-rds-vault --region us-east-1
2aws backup list-backup-jobs --by-resource-arn arn:aws:rds:us-east-1:111122223333:db:prod-orders-db
3aws backup list-copy-jobs --by-state COMPLETED --region us-east-1

Then perform an actual restore into an isolated non-production VPC. Never into the production subnet. Record the restored instance identifier and the restoration duration. This is the real completion check. If you cannot produce a working restored database from a locked recovery point, the control does not meet the requirement yet, no matter what the console shows.

#Common Traps

SymptomLikely causeFix
Copy job fails on an access-denied condition against the destination keyDestination region KMS key policy does not grant the backup service principal kms:CreateGrant and kms:DecryptUpdate the destination key policy to include the backup service principal and the source account
Restore test fails on an access-denied conditionAWSBackupServiceRolePolicyForRestores is not attachedAttach the managed policy and retry the restore
First scheduled job silently excludes the RDS instanceTag applied after the plan’s evaluation window, with nothing flaggedRe-tag, wait a full evaluation cycle, confirm via list-backup-jobs
Plan update rejected on a retention changeMaxRetentionDays set lower than the age of existing recovery pointsRaise the max retention value, or let existing points expire before tightening it
Vault becomes immutable earlier than plannedGrace period expired before testing finishedNone available. This is exactly what Rollback below covers

#Rollback

If you are still inside the grace period, rollback is straightforward:

1aws backup delete-backup-vault-lock-configuration 
2  --backup-vault-name prod-rds-vault 
3  --region us-east-1

Confirm with describe-backup-vault that the lock configuration block is gone. To remove the cross-region copy action without touching the lock, edit the plan JSON

to drop the CopyActions array and run aws backup update-backup-plan with the revised document. To decommission the plan while keeping recovery points intact, delete the backup selection first, then the plan. The vault and its stored recovery points are unaffected by plan deletion.

If the grace period has already expired and the vault has converted to compliance mode, say so plainly to your manager rather than trying to work around it quietly. There is no API rollback at that point. Recovery points can only be deleted once each one individually passes its MinRetentionDays. The remaining avenue is an AWS Support exception request, which is not guaranteed and is reserved for extraordinary documented circumstances, not routine engineering mistakes. This is exactly why the grace period exists, and exactly why you test in governance mode against a disposable vault before ever pointing this procedure at production.

#Operational Summary

The deliverable for this task is a locked, cross-region-replicated backup vault protecting a named production RDS instance, backed by documented evidence: the vault ARNs and encryption key ARNs, the backup plan and selection IDs, at least one completed backup job and one completed cross-region copy job, the lock configuration output with its grace period expiry date, and a successful restore into an isolated non-production environment with a recorded duration. File all of it against the CAB ticket number before you close the change.

Vault Lock is one of the few controls in AWS where the safe default is deliberate slowness. Test in governance mode first. Prove the restore actually works. Get sign-off from your change board. Only then let the grace period run out into compliance mode. Treat the irreversibility as the feature it is meant to be, not as a risk you talked yourself into, and only after you have personally verified the restore path end to end.

#Learning Objectives

By the end of working through this procedure, you should be able to distinguish governance mode from compliance mode in practical terms, not just definitionally: you should be able to state, from your own testing, exactly what breaks and what does not when a lock configuration is deleted versus when the grace period has already lapsed.

You should also be able to explain the cross-region copy dependency chain: source vault key, destination vault key, the AWS Backup service role, and the specific KMS actions each side needs, because a gap anywhere in that chain produces the access-denied failure mode described in the Common Traps table, and diagnosing it quickly depends on understanding the chain rather than pattern-matching the error text.

A further objective is being able to produce audit-ready evidence without being told what to collect each time: vault ARNs, key ARNs, plan and selection IDs, job IDs in both COMPLETED and RUNNING states, and a restore duration, assembled as a matter of habit rather than as a checklist read off a ticket template.

  • Differentiate governance-mode and compliance-mode behaviour through direct testing rather than documentation alone
  • Trace the KMS permission chain required for a cross-region backup copy to succeed
  • Assemble CAB-ready evidence for a locked, replicated backup configuration without prompting

#Worked Example

The article instructs you to fix the destination KMS key policy when a copy job fails on access-denied, without showing the policy statement itself. In practice, the destination key policy needs a statement granting the AWS Backup service principal, backup.amazonaws.com, the actions kms:CreateGrant, kms:DescribeKey, and kms:Decrypt, scoped with a condition requiring kms:ViaService to match the destination region's backup service endpoint. Without the kms:ViaService condition, the grant is broader than necessary and will likely be flagged in a security review even after the copy job succeeds.

Before editing the key policy, retrieve and save the existing document with aws kms get-key-policy –key-id SECONDARY-KEY-ID –policy-name default –region us-west-2, redirecting the output to a local file. This is your rollback artefact. Only after that file exists should you apply the revised policy with aws kms put-key-policy –key-id SECONDARY-KEY-ID –policy-name default –policy file://revised-policy.json –region us-west-2.

Validate by re-running the failed copy job manually rather than waiting for the next scheduled window: aws backup start-copy-job with the same source recovery point ARN and destination vault ARN used previously. Poll aws backup describe-copy-job until State reads COMPLETED. If it still fails, the fault is elsewhere in the chain, most likely the service role's managed policies checked in Step 3, not the key policy.

Rollback, if the new policy statement causes an unrelated permission regression elsewhere, is to reapply the saved original document with the same put-key-policy call, then re-validate with get-key-policy that the reverted JSON matches the saved file byte for byte before considering the rollback complete.

#Practice Exercise

Using a disposable, non-production vault only, build a monitoring layer the base procedure does not include: an EventBridge rule matching AWS Backup job state changes, filtered to state DENIED and FAILED, with an SNS topic as the target. This closes the gap where, as the article notes, a missed tagging window fails silently with no error surfaced anywhere.

Create the SNS topic first, subscribe your own email address, and confirm the subscription before wiring anything else, since an unconfirmed subscription will silently drop notifications in exactly the same way the article warns about elsewhere. Then create the EventBridge rule with an event pattern matching source aws.backup and detail-type Backup Job State Change, targeting the SNS topic ARN.

Validate by manually triggering a failure condition in your disposable environment only, such as pointing a backup selection at a resource ARN with an intentionally revoked IAM role, and confirming the SNS notification arrives within a few minutes. Then restore the correct role and confirm the next manual job succeeds without alerting.

Rollback is deleting the EventBridge rule, deleting the SNS subscription and topic, and confirming with aws events list-rules and aws sns list-topics that neither resource remains, before discarding the disposable vault itself with aws backup delete-backup-vault once it holds no recovery points.

Evidence trail

Sources and verification

Primary documentation and external technical references used in this article.

  1. 01AWS's Backup Vault Lock documentationdocs.aws.amazon.com
  2. 02cross-region backup copy guidedocs.aws.amazon.com
Priya Nair

Priya Nair

Graduate Track editor

Priya Nair is KBY Technologies’ Graduate Cloud and Automation Editor.

Published
View Profile
Reader Interaction

Comments

Add a thoughtful note on Locking Down Production RDS Backups with AWS Backup Vault Lock. Comments are checked for spam and held for moderation before appearing.

Loading comments...

Discover more

Learn More About KBY

Was this useful?

Build practical engineering skills.

Receive new lessons, learning paths, practical exercises and early-career guidance.