Working with storage for Amazon RDS DB instances
To Nha Notes | Feb. 6, 2024, 10:37 a.m.
- Amazon EBS volumes for RDS storage: RDS uses Amazon EBS volumes to store database and log data, and stripes across multiple volumes for better performance and reliability1.
- Storage scaling modifications: RDS can scale the storage size, Provisioned IOPS, or storage type of a DB instance without affecting the database operations, by using the Amazon EBS elastic volume modification feature.
- I/O-intensive storage modifications: Some older RDS storage volumes or larger storage sizes require a different process for storage scaling, which involves copying the data and consuming more I/O resources2. This can affect the performance and availability of the DB instance.
Amazon RDS provides four storage types:
General Purpose SSD storage: General Purpose SSD storage offers cost-effective storage that is ideal for a broad range of workloads running on medium-sized database instances. Amazon RDS offers two types of General Purpose SSD storage: gp2 storage and gp3 storage.
gp2 storage
gp2 storage are best suited for development and testing environments. The I/O capacity of the instance is based on the instance storage type and size. If the DB instance is configured with a gp2 volume, the baseline IOPS capacity is 3x the GiB storage. If the DB instance has allocated a 100-GiB gp2 volume, the baseline IOPS capacity is 300. The more storage you provision, the higher the IOPS capacity. In addition to baseline IOPS capacity, gp2 volumes also deliver burst capacity up to 3,000 IOPS for extended periods of time. The burst feature is limited to volumes equal to or less than 1 TiB of storage. DB instances for MySQL, MariaDB, Oracle, and PostgreSQL can be configured with up to 64 TiB of storage, but the max baseline IOPS is limited from 100 to 16,000 IOPS per volume. So, a gp2 volume of 5.34 TiB or more delivers the same baseline: 16,000 IOPS. For 4 volumes of gp2 storage, the max baseline IOPS is 64,000 IOPS.
gp3 storage
gp3 storage volume provides you freedom of customizing storage performance independent of storage capacity. Storage performance is the combination of IOPS and storage throughput. On gp3 volumes, RDS provides a baseline storage performance of 3000 IOPS and 125 MiB/s throughput. IOPS and storage throughput don’t depend on the storage size, but they are related to each other. In MariaDB, MySQL, and PostgreSQL for storage size more than 400 GB and for RDS Oracle with storage size more than 200 GB, RDS uses 4 volumes instead of 1 due to volume striping. Thus these baseline storage performance increases to 12,000 IOPS and 500 MiB/s.
io1 storage
Provisioned IOPS (io1) storage is a storage type that delivers predictable performance, and consistently low latency. It’s optimized for online transaction processing (OLTP) workloads that require consistent performance. If your production workload requires high OLTP, and fast, consistently high throughput performance, you should configure your DB instance with io1 volumes. Compared to gp2/gp3 volumes, which deliver a maximum baseline of 64,000 IOPS for 4 volumes, io1 volumes can deliver up to 256,000 IOPS for DB instances for MySQL, MariaDB, Oracle, and PostgreSQL and up to 64,000 for SQL Server instances. If you find the pattern of IOPS usage consistently going beyond more than 64,000, you should modify the DB instance and change the storage type from gp2/gp3 to io1.
Magnetic
Amazon RDS also offers magnetic storage, but it’s not suitable for an OLTP workload requiring consistent I/O performance and low latency. The Magnetic storage type is not recommended for I/O-intensive workloads because the maximum storage is less than that of gp2 or io1. The IOPS capacity is also limited to a maximum of 1,000 IOPS.
Upgrading the storage file system for a DB instance
Most RDS DB instances offer a maximum storage size of 64 TiB for MySQL and MariaDB databases. However, some older 32-bit file systems have lower storage capacities. To determine the storage capacity of your DB instance, you can use the describe-valid-db-instance-modifications AWS CLI command.
If RDS detects that one of your DB instances is running an older file system (one that has a 16 TiB storage size, a file size limit of 2 TiB, or non-optimized writes), the RDS console informs you that your file system configuration is eligible for an upgrade. You can check the upgrade eligibility of your DB instance on the Storage panel of the DB instance details page.
If your DB instance is eligible for a file system upgrade, you can perform the upgrade in one of two ways:
-
Create a blue/green deployment and specify Upgrade storage file system configuration. This option upgrades the file system in the green environment to the preferred configuration. You can then switch over the blue/green deployment, which promotes the green environment to be the new production environment. For detailed instructions, see Creating a blue/green deployment.
-
Create a DB instance read replica and specify Upgrade storage file system configuration. This option upgrades the file system of the read replica to the preferred configuration. You can then promote the read replica to be a standalone instance. For detailed instructions, see Creating a read replica.
Upgrading the storage configuration is an I/O-intensive operation and leads to longer creation times for read replicas and blue/green deployments. The storage upgrade process is faster if the source DB instance uses Provisioned IOPS SSD (io1) storage and you provisioned the green environment or read replica with an instance size of 4xlarge or larger. Storage upgrades involving General Purpose SSD (gp2) storage can deplete your I/O credit balance, resulting in longer upgrade times. For more information, see Amazon RDS DB instance storage.
During the storage upgrade process, the database engine isn't available. If the storage consumption on your source DB instance is greater than or equal to 90% of the allocated storage size, the storage upgrade process will increase the allocated storage size by 10% for the green instance or read replica.
Modifying settings for Provisioned IOPS SSD storage
You can modify the settings for a DB instance that uses Provisioned IOPS SSD storage by using the Amazon RDS console, AWS CLI, or Amazon RDS API. Specify the storage type, allocated storage, and the amount of Provisioned IOPS that you require. The range depends on your database engine and instance type.
Although you can reduce the amount of IOPS provisioned for your instance, you can't reduce the storage size.
In most cases, scaling storage doesn't require any outage and doesn't degrade performance of the server. After you modify the storage IOPS for a DB instance, the status of the DB instance is storage-optimization.
Note
Storage optimization can take several hours. You can't make further storage modifications for either six (6) hours or until storage optimization has completed on the instance, whichever is longer.
References
Working with storage for Amazon RDS DB instances - Amazon Relational Database Service