Terraform AWS [aws-organization-backup]

This module implements a standard AWS Backup setup using AWS Organizaion backup policies for enforcement.

Any supported and enabled resource with tag BackupEnabled and value True will be included.

The module is currently tested for all scenarios except cross-region and cross-account combined. This probably just works or should be easy to implement.

All modules should work but currently only EBS, RDS and S3 are tested. EFS, Virtual Machines and DynamoDB might need some work because KMS is supported.

*** Make sure you GUARD or BACKUP your KMS CMK keys as AWS Backup mostly uses the original resource KMS CMK key for encrypting the backups. The best approach is to block kms:ScheduleKeyDeletion in an SCP.

Simultaneous cross-region and cross-account is not supported for RDS, Aurora, Neptune and DocumentDB.

How does it work

AWS Backup works by copying targeted resource's data into s3 storage on a schedule. You can select which resources you want to back up with the use of tags. (explained further in ./plan.md). The data can be backed up to other regions or AWS accounts. AWS Backup integrates with other AWS services, such as AWS Identity and Access Management (IAM) for authentication and Amazon CloudWatch for logging and monitoring.

Known (major) limitations

The module is currently tested for all scenarios except cross-region and cross-account combined. This probably just works or should be easy to implement.

Requirements

• This module requires at least the following Terraform configuration on the management account.

resource "aws_organizations_organization" "this" {
  aws_service_access_principals = [ "backup.amazonaws.com" ]

  enabled_policy_types = [ "BACKUP_POLICY" ]
}

• When enable_external_vault is true then make sure that the provider aws.external_vault is set and from the same region as the AWS account.

• When the backup_vault_kms_key_arn is in another account make sure that any providers that create vaults have access to this KMS key. Required permissions:

  • kms:CreateGrant
  • kms:GenerateDataKey
  • kms:Decrypt
  • kms:RetireGrant
  • kms:DescribeKey

This requirement can be automated once Terraform aws_kms_grant supports service principals. See issue 13994 for this (please upvote!).

• All accounts with vaults must have the AWSServiceRoleForBackup service linked role. This can be created / imported in Terraform with:

resource "aws_iam_service_linked_role" "backup_service_linked_role" {
  aws_service_name = "backup.amazonaws.com"
}

• This modules requires KMS access for the role role/aws-service-role/backup.amazonaws.com/AWSServiceRoleForBackup. Our KMS key module has these policies.

• Enable all resources for each region in the management account to make sure that resources are included.

resource "aws_backup_region_settings" "this" {
  resource_type_opt_in_preference = {
    "Aurora" = true,
    "CloudFormation" = true,
    "DocumentDB" = true,
    "DynamoDB" = true,
    "EBS" = true,
    "EC2" = true,
    "EFS" = true,
    "FSx" = true,
    "Neptune" = true,
    "RDS" = true,
    "Redshift" = true,
    "S3" = true,
    "Storage Gateway" = true,
    "Timestream" = true,
    "VirtualMachine" = true
  }

  resource_type_management_preference = {
    "DynamoDB" = true
    "EFS"      = true
  }
}

• S3 buckets with KMS encrypted objects require that the backup role outputed with backup_role_arn have Decrypt, DescribeKey permissions on the KMS key.

Example below with KMS grant.

resource "aws_kms_grant" "s3_appdata" {
  name              = "aws_backup_${var.name}_s3_appdata"
  key_id            = data.terraform_remote_state.acme.outputs.acme_kms_key_arn
  grantee_principal = module.organization_backup.backup_role_arn
  operations        = ["Decrypt", "DescribeKey"]
}

• RDS instances with KMS encrypted snapshots require that the backup role outputed with backup_role_arn have DescribeKey, Decrypt, ReEncryptFrom, ReEncryptTo, CreateGrant, RetireGrant permissions on the KMS key.

resource "aws_kms_grant" "s3_appdata" {
  name              = "aws_backup_${var.name}_s3_appdata"
  key_id            = data.terraform_remote_state.acme.outputs.acme_kms_key_arn
  grantee_principal = module.organization_backup.backup_role_arn

  operations        = ["DescribeKey", "Decrypt", "ReEncryptFrom", "ReEncryptTo", "CreateGrant", "RetireGrant"]
}

Known issues

Initial creation could results in errors like below. Retry again to resolve.

╷
│ Error: error creating Backup Vault Lock Configuration (name): AccessDeniedException:
│ status code: 403, request id: 44cfe1e4-7aab-4c95-b142-9e600b278916
│
│ with module.organization_backup.module.backup_vault_external[0].aws_backup_vault_lock_configuration.this,
│ on modules/aws-organization-backup/backup_vault/main.tf line 10, in resource "aws_backup_vault_lock_configuration" "this":
│ 10: resource "aws_backup_vault_lock_configuration" "this" {
│
╵

Sometimes it looks like AWS Backup is not working but it simply could take hours(!) before something happens.

Enable AWS EventBridge rules on aws.backup to closely monitor events and issues since you can also see CloudTrail events. These events also tend to happen long before anything is visible in the web console.

Usage

To use this module see the ./examples directory for the 3 main supported and tested workflows with EBS, S3 and RDS.

ALWAYS make sure you see your resources in 'Protected Resources' before assuming that the backup plan is correctly configured.

Future work

• Combined cross-account and cross-region. Probably requires seperate KMS keys.
• Automatic handling of setting up KMS access under different configurations (e.g. KMS per vault location, KMS shared in source vault account, KMS shared in destination vault account).

Providers

Name	Version
aws	>4.27
aws.external_vault	>4.27
aws.management	>4.27

Modules

Name	Source	Version
backup_vault	./backup_vault	n/a
backup_vault_external	./backup_vault	n/a
iam_role	../identity_and_access_management/iam_role/	n/a

Resources

Name	Type
aws_backup_global_settings.this	resource
aws_backup_vault_policy.source_account_to_destination_account_vault_access	resource
aws_kms_grant.kms_grant_external_vault	resource
aws_organizations_policy.this	resource
aws_organizations_policy_attachment.this	resource
aws_caller_identity.current	data source
aws_caller_identity.external_vault	data source
aws_default_tags.current	data source
aws_iam_policy_document.source_account_to_destination_account_vault_access	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
backup_vault_kms_key_arn	Backup vault KMS key ARN to be used for internal and optional external vault. Required.	string	n/a	yes
enable_external_vault	This will create a mimic vault in the account provided by aws.external_vault. If set to false then aws.external_vault can be set to any account as it's not used.	bool	n/a	yes
external_backup_vault_kms_key_arn	External backup vault KMS key ARN to be used for external vault. If not set then value taken from var.backup_vault_kms_key_arn is used. This variable must be set to support cross-region setups.	string	null	no
immutable_vault	Make the local and (optional) external vault immutable to prevent deletion. Immutable vaults can never be deleted after 90 days.	bool	true	no
name	Name for several resources to allow this module to be reused within the same account. Must also be unique for any vaults in external accounts. Required	string	n/a	yes

Outputs

Name	Description
backup_role_arn	n/a