Amazon EBS: How to Get More from Your Elastic Block Store

Amazon Elastic Block Store (EBS) is the most widely used cloud storage service for Amazon Web Services (AWS) — and with good reason. It is designed specifically for use with EC2 (Elastic Cloud Compute), is suited to a wide variety of use cases, offers several different volume types, and boasts an extensive range of options and features.

However, it also represents a significant chunk of your monthly Amazon Web Services bill. Therefore, it’s essential you use EBS wisely by striking the right balance between cost and performance, and eliminating unnecessary waste on storage resources.

To make the most of your organization’s cloud spend and leverage resources efficiently, it’s important to understand Amazon EBS. This guide will:

• Break down the basics of EBS vs. S3 storage on AWS
• Explain the different EBS volume types
• Discuss some of the key features of EBS
• Provide a few monitoring tips to help you reduce EBS expenditure

But first, let’s take a quick recap of what Amazon EBS actually is.

What Is Amazon EBS?

EBS is one of Amazon’s core block storage options designed for use with EC2 instances. It is gaining increasing favor over ephemeral block storage alternative Amazon EC2 instance store.

Unlike instance store, data stored on EBS is persistent, allowing you to stop and restart instances without loss of data. This can help you reduce your cloud costs, as you can shut down instances whenever applications aren’t needed. What’s more, it offers faster boot times and a wider range of features. EBS is also ideal for encrypting data and/or retaining it for longer periods.

In line with block-level storage in general, EBS comes in the form of mountable fixed-sized raw storage disks or volumes, which you can format with the file system of your choice. Originally, you could only dedicate an EBS volume to one instance at any one time, but AWS enabled Multi-Attach where multiple hosts in the same Availability Zone can connect to one EBS volume, for read and write, which opens up new opportunities.

EBS vs. S3: How Do They Compare?

As the alternative block storage system to Instance Store, EBS shares many similar characteristics with its ephemeral counterpart. However, it is significantly different from Amazon’s object storage service, S3.  

EBS gives developers a way to format volumes with a familiar file system, so they can build application environments in the cloud based on traditional on-premise architecture. By contrast, S3 is general-purpose storage geared towards applications that are either born in or rearchitected for the cloud.

You can only access Amazon EBS through an EC2 instance, whereas objects stored on S3 are available to anyone anywhere, provided they have the object path name (essentially a web address) or access via API.

But one of the most significant differences is cost. EBS is more expensive than S3. What’s more, unlike S3, it is provisioned storage. This means you specify the size in advance. So make sure you pay careful attention to your volume sizes to avoid paying for storage you don’t need. This is particularly important at the outset, as there’s no easy way to resize volumes on the fly without interruption to availability.

However, with so many applications designed for use with block storage, Amazon EBS is suited to a much wider range of use cases, including both relational and NoSQL databases and enterprise applications such as SAP and Microsoft Exchange. Such applications cannot run on S3 alone.

EBS Volume Types

Amazon offers a choice of several different categories of EBS volume. As each option is tailored to different types of workload, it’s important to understand the differences between them to get the best balance of cost and performance.

Solid State Drives (SSD)

SSDs are capable of delivering substantially higher IOPS (I/O operations per second) than traditional magnetic drives and perform best when handling very frequent but small transactions. However, they cost significantly more, at more than twice the price of hard disk drives.

EBS General Purpose SSD

Types: gp2 (default) and gp3

Volume Size: 1GB–16TB

Max. IOPS/Volume: 16,000

Max. Throughput/Volume: 250 MB/s (gp2) and 1,000 MB/s (gp3)

Price Range: Some free tier (125 MB/s free) up to $0.08/GB-month

The default EBS option is designed for general use. This type of volume is ideal for virtual desktops, medium-sized single-instance databases like Microsoft SQL Server and Oracle, latency-sensitive interactive applications, dev/test environments, and boot volumes. 

EBS Provisioned IOPS SSD

Types: io1, io2, and io2 Block Express

Volume Size: 4GB–16 TB (io1 and io2) and 4GB–64TB (io2 Block Express)

Max. IOPS/Volume: 64,000 (io1 and io2) and 256,000 (iop2 Block Express)

Max. Throughput/Volume: 1,000 MB/s (io1 and io2) and 4,000 MB/s (io2 Block Express)

Price Range: $0.032/provisioned IOPS-month for greater than 64,000 IOPS to $0.125/GB-month

This is the storage class with the highest performance capabilities of all EBS volume types. However, it’s also the most expensive at $0.125 per GB-month of provisioned storage. AWS recommends using the io1 and io2 EBS volumes for I/O-intensive NoSQL and relational databases. The io2 Block Express can handle the largest, most I/O-intensive mission critical deployments of NoSQL and relational databases; it also has a sub-millisecond latency.

Hard Disk Drives (HDD)

HDDs are both less expensive and less responsive than SSDs, with considerably lower IOPS performance (maximum of just 500 IOPS/volume for both). However, they offer comparable throughput levels to SSD and are ideal for applications that read and write large chunks of data or for more infrequently accessed workloads.

EBS Throughput Optimized HDD

Type: st1

Volume Size: 125GB–16TB

Max. IOPS/Volume: 500

Max. Throughput/Volume: Up to 500 MB/s

Price: $0.045/GB-month

Use cases for this low-cost general magnetic storage option include log processing, data warehousing, and big data. It is ideal for frequently accessed and throughput-intensive workloads with large datasets or I/O sizes.

EBS Cold HDD

Type: sc1

Volume Size: 125GB–16TB

Max. IOPS/Volume: 250

Max. Throughput/Volume: Up to 250 MB/s

Price: $0.015/GB-month

EBS Cold HDD offers an extremely inexpensive block storage option for applications that require only occasional or unpredictable access. These volumes can burst up to 80 MB/s per TB, with a baseline throughput of 12 MB/s per TB up to 250 MB/s per volume. 

Key EBS Features

EBS offers many useful features, some of which are included in the price while others come at an additional cost:

Automatic Replication

AWS automatically replicates your EBS volumes within the same Availability Zone (AZ). This helps prevent data loss in the event of hardware failure. However, this may not be sufficient to safeguard your data at enterprise scale, as the frequency of volume failures within a large cloud environment will inevitably be higher. What’s more, this doesn’t protect you if an entire AZ goes down; ultimately, your EBS costs will also include the storage needed for your failover architecture—either across AZs or Regions, depending on your availability requirements.

Data Encryption

EBS supports data encryption to AES 256-bit, which can help users to meet compliance standards, such as HIPAA, PCI and NIST. You can encrypt data at rest inside the volume, all snapshots created from the volume, and all disk I/O. It is available to certain EC2 instance types and comes at no additional cost. 

Dedicated Network Capacity (EBS Optimization)

EBS Optimization provides dedicated bandwidth between your EC2 instances and EBS volumes. It is designed to improve I/O and throughput performance by minimizing contention between competing processes that share EC2 resources. Most instances are optimized by default. For those that are not, EBS Optimization is available at an additional charge per hour.

EBS Snapshots

Snapshots are point-in-time backups of your EBS volumes to S3. Backups are incremental, which means only the blocks that have changed since your last snapshot are saved. Not only that, but snapshots also leverage data compression and the lower storage charges on S3, making it a cost-effective method of backing up EBS volumes. You can also use EBS snapshots to create template machine images.

Amazon EBS Optimization Tips

How can you get more from EBS? Here are our tips for managing, monitoring, and optimizing your Elastic Block Store on AWS.

1. Know Your EBS Snapshot Costs

As EBS snapshots are incremental, each individual backup is relatively small. However, this means snapshot costs can creep up slowly—until, before you know it, your charges have spiraled out of control. This can be a particular problem if you automate your snapshots, where it’s essential to build in a deletion schedule for older backups.

Snapshots are backed up to S3 but show up in the EBS section of your monthly AWS bill. Pricing for EBS snapshot storage is slightly different from standard S3 rates at $0.05 per GB-month of data stored.

2. Pay Attention to Backup Size

The monthly cost of storing your snapshots is dictated by the size of your initial backup and the total size of your incremental backups over your retention period.

Your initial backup size will be equal to the amount of storage actually used on your EBS volume, with a slight reduction to allow for data compression. The total size of your incremental backups will depend on how frequently the data on your volume changes over time and how long you choose to retain your snapshots.

Data on a production server typically changes by about 3% per day. Multiply that by 30 days, and you arrive at a figure of 90% per month. So, based on daily snapshots over a retention period of 30 days, you’ll likely need about twice the size of your initial snapshot to accommodate both your first and subsequent backups.

3. Beware of Unattached Volumes and Orphan Snapshots

When you terminate an EC2 instance, by default, only EBS root device volumes are automatically deleted. By contrast, attached volumes and snapshots persist on EBS and S3 respectively.

To avoid running up unnecessary storage costs, you should delete unattached volumes and orphan snapshots when you no longer need them.

Note: make sure you tag EBS resources so you can keep track of them throughout their lifetime. You can also simplify the task of maintaining your EBS inventory by using third-party cloud management tools that automatically clean up unused resources for you.

4. Calculate Your I/O and Throughput Utilization

Amazon CloudWatch provides a number of standard metrics for checking the operational behavior of your EBS volumes. The most important of these are:

• VolumeReadOps: Total number of read operations per specified period
• VolumeWriteOps: Total number of write operations per specified period
• VolumeThroughputPercentage (Provisioned IOPS SSD only): IOPS delivered as a percentage of total IOPS provisioned for volume

In the case of IOPS, CloudWatch doesn’t provide a metric directly. So you’ll need to calculate it yourself. By default, CloudWatch collects data every five minutes. So you’d need to add together your read and write metrics and divide by 300 seconds as follows:

• IOPS = (VolumeReadOps + VolumeWriteOps) / 300

Your IOPS and throughput percentage metrics will help you determine whether you’re under- or overutilizing your EBS volumes. Using baseline IOPS, baseline throughput and bursting capabilities as a guide, you may need to resize them to get a better balance of cost and performance. Alternatively, you could consolidate a set of smaller attached volumes into fewer larger ones or consider switching to another volume type.

With Provisioned IOPS SSD in particular, you should pay close attention to your IOPS utilization, as you pay for your provisioned IOPS directly.

5. Monitor Disk Utilization

Disk utilization is not a standard metric that you can monitor in CloudWatch. So, it’s not easy to check whether you’re making good use of the storage you’ve actually provisioned. Nevertheless, it’s still possible to monitor disk space as a custom metric by using a CloudWatch monitoring script, which you can download from the AWS sample code library.

Again, with Provisioned IOPS SSD, it’s particularly important to monitor your disk utilization as, unlike with the other three volume types, you can scale storage independently of IOPS, giving you more scope for precise optimization.

6. Follow EBS Best Practices

Best practices, such as tagging resources, monitoring utilization, and cleaning up unattached volumes and orphan snapshots, are key to getting a bigger bang for your buck from your EBS deployments.

You should also minimize the time-consuming and costly manual processes involved in maintaining your EBS storage. For example, you could implement a solution that automates your snapshot schedule, saving you time and ensuring you never miss a backup.

Finally, you should integrate migration processes into your storage lifecycle, such as rotating instance logs from EBS to S3 or Glacier. By offloading this data to less expensive storage, you can make even further savings on your monthly cloud bills. Consider leveraging Intelligent Tiering to let AWS handle the data residency, balancing access time and cost.

Comprehensive Cloud Management to Optimize Cloud Spend

Savvy enterprises have realized that managing a rapidly scaling environment in the cloud can be a struggle. Cloud management platforms like CloudCheckr CMx can help simplify and streamline the process of optimizing workloads in a dynamic, ephemeral environment. With CloudCheckr, organizations gain visibility and control across resources, with self-healing automation for critical tasks to keep infrastructure safe and stable.

Want to know what CloudCheckr can do for your AWS solutions?

CloudCheckr provides visibility, intelligence and automation to keep your environment optimized for performance, reliability and spend. Ready to see CloudCheckr CMx in action? Schedule a free 30-minute demo.