Export RDS database snapshots manually to S3

To Nha Notes | Nov. 18, 2022, 9:49 a.m.

RDS already has an 'Export to S3' button. Alternatively you can use AWS Data Migration Service (DMS) to export the data to S3 in real-time.]

You can export DB cluster snapshot data to an Amazon S3 bucket. The export process runs in the background and doesn't affect the performance of your active DB cluster.

When you export a DB cluster snapshot, Amazon Aurora extracts data from the snapshot and stores it in an Amazon S3 bucket. You can export manual snapshots and automated system snapshots. By default, all data in the snapshot is exported. However, you can choose to export specific sets of databases, schemas, or tables.

By exporting an Amazon RDS DB cluster, you can extract data from that DB cluster in an automated way. The extracted data is stored in an Amazon S3 bucket in your account in an Apache Parquet format, a compressed, consistent format that you can query.

By exporting RDS DB cluster data to a Parquet format in S3, you can reduce the cost of retaining long-term data and make the data more accessible.

Pricing

Charge per GB of snapshot size:$0.012

For example, suppose you have a 100GB snapshot and you use filtering to select a 10GB table from this snapshot to export to Amazon S3. To export this data you would pay 100GB * $0.012 per GB of snapshot size. Subsequent exports of data from the same snapshot are not incremental.

Overview of exporting snapshot data

You use the following process to export DB snapshot data to an Amazon S3 bucket. For more details, see the following sections.

1. Identify the snapshot to export.

   Use an existing automated or manual snapshot, or create a manual snapshot of a DB instance.

2. Set up access to the Amazon S3 bucket.

   A bucket is a container for Amazon S3 objects or files. To provide the information to access a bucket, take the following steps:

   1. Identify the S3 bucket where the snapshot is to be exported to. The S3 bucket must be in the same AWS Region as the snapshot. For more information, see Identifying the Amazon S3 bucket for export.

   2. Create an AWS Identity and Access Management (IAM) role that grants the snapshot export task access to the S3 bucket. For more information, see Providing access to an Amazon S3 bucket using an IAM role.

3. Create a symmetric encryption AWS KMS key for the server-side encryption. The KMS key is used by the snapshot export task to set up AWS KMS server-side encryption when writing the export data to S3. For more information, see Encrypting Amazon Aurora resources.

   The KMS key is also used for local disk encryption at rest on Amazon EC2. In addition, if you have a deny statement in your KMS key policy, make sure to explicitly exclude the AWS service principal export.rds.amazonaws.com.

   You can use a KMS key within your AWS account, or you can use a cross-account KMS key. For more information, see Using a cross-account AWS KMS key.

4. Export the snapshot to Amazon S3 using the console or the start-export-task CLI command. For more information, see Exporting a snapshot to an Amazon S3 bucket.

5. To access your exported data in the Amazon S3 bucket, see Uploading, downloading, and managing objects in the Amazon Simple Storage Service User Guide.

References

https://docs.aws.amazon.com//AmazonRDS/latest/AuroraUserGuide/aurora-export-snapshot.html

https://docs.aws.amazon.com//AmazonRDS/latest/AuroraUserGuide/export-cluster-data.html

https://dev.to/aws-builders/export-database-snapshots-manually-to-s3-export-s3-content-in-glue-data-catalog-using-crawler-for-tables-fetched-in-athena-5ef6