Performance of burstable CPU VM instances

Last updated on

Burstable CPUs VM instances appeared on Amazon EC2 for the first time in 2014. Since then, they have become ubiquitous at major providers in their entry level plans. The implementation varies between providers but the common denominator is that the user gets a fraction of one or more cpu cores to use as a baseline and is additionally allowed short bursts of CPU usage to 100%. The lower the average cpu usage is, the longer the CPU bursts can last.

The main advantage of those plans is that they more affordable than others. Compare the EC2 t3 medium (burstable, 2 cores, 4GB) at $0.0416 per hour to the closest fixed cpu allocation VM, the c5 large (2 burstable cores, 4GB): at $0.085 per hour, it will cost you double the price for comparable specs. See Amazon EC2 pricing page.

We have tested 10 of those plans at vpsbenchmarks.com from Amazon EC2, Lightsail, Google Compute Engine, Alibaba Cloud and Microsoft Azure. Here is the list of plans with their price and specifications:

Name Cost per month Number of cores Memory CPU Baseline
Alibaba Cloud - ecs-t5-c1m1-large $25.00 2 2.0 GB 15%
Amazon EC2 - t2-medium $37.00 2 4.0 GB 20%
Amazon EC2 - t2-micro $11.64 1 1.0 GB 10%
Amazon EC2 - t2-small $28.00 1 2.0 GB 20%
Amazon EC2 - t3-medium $37.00 2 4.0 GB 20%
Amazon Lightsail - 1GB 1core $5.00 1 1.0 GB 10%
Amazon Lightsail - 2GB 1core $10.00 1 2.0 GB 25%
Amazon Lightsail - 4GB 2cores $20.00 2 4.0 GB 35%
Google Compute Engine - g1-small $27.00 1 1.7 GB 50%
Microsoft Azure - B1MS $19.25 1 2.0 GB 20%
Microsoft Azure - B2s $31.00 2 4.0 GB 40%

Using Endurance tests to evaluate burstable VMs

We want to know which ones of those plans can do the most work over the long term relative to their prices. The most relevant test to judge of the long term performance of a burstable VM is the "Endurance Test". It consists of a sustained CPU usage script that runs for 24 hours, sorts and shuffles a dictionary in a loop and maintains a 50% cpu usage for the entire time. While the script runs, the number of iterations (sort+shuffle) is reported and recorded every 10 minutes.

Typically, this is what the number of iterations per hours looks like, hour after hour, over a day:
Num iterations chart
See the EC2 t2 medium endurance page for example.

What you see is the VM going full speed for a few hours while cpu credits (or equivalent) are still available and then the throughput drops sharply at the same time as CPU steal goes up dramatically.

In the end, the number that matters and that we will use to compare the performance of those plans is the average number of iterations completed per hour over the 24 hours hours of the test. This number for all burstable plans is displayed on this endurance comparison page.

Endurance Burstable Perf Results

Observations

  • Alibaba Cloud t5-c1m1-large comes out as the winner having completed the highest number of operations over 24 hours.
  • EC2 t2 medium and Lightsail 4GB have identical performance, they most likely are the same type of instance.
  • Azure is in the middle of the pack.
  • Lightsail 1GB (cheapest in the Lightsail range) is the worst performer. The short duration of the "burst" is visible in the endurance run details, it can only manage 3 hours at high speed.

Performance adjusted for price

But that's not the whole story, price should also be a factor in choosing your cloud server. The following table shows the number of iterations per hour divided by the cost of the instance. In addition, the last column has the number of hours in the test before the instance gets throttled.

Name Cost per month Hours before throttle Iterations Iterations per dollar
Alibaba Cloud - ecs-t5-c1m1-large $25.00 9 3185 127
Amazon Lightsail - 2GB 1core $10.00 17 1095 109
Amazon Lightsail - 4GB 2cores $20.00 17 2168 108
Amazon Lightsail - 1GB 1core $5.00 4 403 80
Microsoft Azure - B2s $31.00 18 2124 68
Microsoft Azure - B1MS $19.25 18 1143 59
Amazon EC2 - t2-medium $37.00 17 2184 59
Google Compute Engine - g1-small $27.00 24 1468 54
Amazon EC2 - t3-medium $37.00 6 1732 46
Amazon EC2 - t2-micro $11.64 5 458 39
Amazon EC2 - t2-small $28.00 17 1017 36

If you're trying to make up your mind between EC2 and Lightsail and performance/cost is your deciding factor, it's clear that you should go with the latter. The Lightsail 1GB plan, that seemed pretty plain when considering exclusively its performance, looks much better once the price is taken into account. That's especially true if you compare it to the EC2 t2 micro (likely the exact same type of instance), you get double the cpu power for every dollar you spend.

A few more observations:

  • The GCP g1 small instance didn't get throttled at all during the 24 hour endurance test. This is because the baseline CPU allocation for this plan is 50%, which is the cpu usage targeted by the endurance test.
  • Surprisingly, EC2 t2 medium performs better than the newer t3 medium for about the same price.

Workloads that are best suited for burstable VMs

Those plans are more affordable than others and they are a good deal for running light traffic web servers. If the system must run some background tasks occasionally, that might work too as long those tasks are not running all day long.

The most common mistake that is made when planning a computing deployment is to allocate resources for peak load 24/7. As a result the machines end up staying idle most of the time and you pay for a lot more than is really needed. With burstable VMs, your capacity grows at peak traffic but gets back to affordable levels outside of those peaks. Some would say this is the best of both worlds.




Be the first to learn about new Best VPS rankings. Subscribe to our newsletter.