Tuesday, March 10, 2009

When is Amazon's EC2 appropriate for your workload?

[UPDATE: The screen-profiles project has been renamed, 'byobu'. The functionality described in this article is now supported there.]

2009 is shaping up to be the Year of Cloud Computing!

Ubuntu Jaunty Jackalope is set to release next month (9.04) and introduce Eucalyptus as an important building block for establishing cloud capabilities in your data center using entirely open source technologies within the Ubuntu distribution.

Meanwhile, the Canonical Server Team is also working diligently to deliver Hardy, Intrepid, and Jaunty images for Amazon EC2, for cloud computing beyond your local data center.

And looking a bit further out, Mark Shuttleworth has an ambitious vision for Ubuntu Karmic Koala, making it easy to deploy applications to the cloud, ready-to-run cloud applicances, and image creation utilities.

We're hoping to position Ubuntu as an ideal platform for your cloud computing needs.

I believe many system adminstrators and IT departments will soon ask themselves:
  • When Amazon's EC2 is appropriate for our workload?
I have written a small utility, called ec2-cost that can help you analyze this question. It is included in Ubuntu in the screen-profiles package.

If you're running Jaunty, simply:
  • sudo apt-get install screen-profiles

Otherwise, you can grab the package for Hardy or Intrepid from:
Then, run:
  • screen
  • touch $HOME/.screen-profiles/ec2-cost
  • Hit F5, then ENTER
In the status bar across the bottom of your screen, you'll see several bits of system status, including an approximate running total cost of your current system. This estimation accounts for the current system uptime, number of processors, inbound, and outbound network activity. It does not yet account for S3 storage charges, or the 10% hike in European instance pricing, but such patches are welcome ;-) You can find the code in:
  • /var/lib/screen-profiles/ec2-cost

Here are a few screen shots...

This little test machine, which lived a total of 4 hours, would be quite affordable in EC2 at ~$0.41:



Let's imagine that I used this system to do some heavy duty number crunching overnight. ~$14.73 might be a pretty good deal. In fact, if the application I'm running is parallelizes well, it might even be less costly (in time or money) to use one of the 8-CPU large instances:



On the other hand, I have determined that EC2 would not be a good model for my always-up production website, DivItUp.com, which has some 400+ days of uptime, at ~$967.13 and counting:


If you want to run the utility outside of screen-profiles, that's possible too:

kirkland@living:~$ /var/lib/screen-profiles/ec2-cost --detail
================================================
Estimated cost in Amazon's EC2 since last reboot
================================================
Network sent: 7.918646 GB @ $0.10/GB
Network recv: 68.167842 GB @ $0.17/GB
Network cost: $8.162954
------------------------------------------------
Uptime: 303 hr @ $0.200000/hr
Uptime cost: $60.600000
------------------------------------------------
Total cost: ~$68.76
================================================

There are certainly other factors that you may need to consider, such as the privacy of your data and whether that belongs in a remote cloud, the scalability of your own hardware versus Amazon's, your own heating/cooling/bandwidth costs, etc. before you can make a fully informed decision.

But hopefully this little utility can help with the initial analysis. And like watching your system load, or memory utilization, it might help you better understand the nature of your server's workloads.

:-Dustin

4 comments:

  1. Nice kob. It really makes us see a better picture of the EC2 scenario.

    ReplyDelete
  2. This would be useful for the sysads who actually manage EC2 instances and not even mindful of the cost and just think its just a few cents per hour. I've had a lot of stories of people using EC2 to run their websites and balk at their monthly bill because it became much more expensive than their former hosts.

    Of course, I've had my fair share of good stories of utilizing EC2 to run 200+ servers for a specific job and destroy them servers after a day or two when the job is finished.

    ReplyDelete
  3. http://aws.typepad.com/aws/2009/03/announcing-ec2-reserved-instances.html

    ReplyDelete
  4. Please correct the address https://launchpad.net/~screen-profiles/+archive/ppa - not valid...

    ReplyDelete

Please do not use blog comments for support requests! Blog comments do not scale well to this effect.

Instead, please use Launchpad for Bugs and StackExchange for Questions.
* bugs.launchpad.net
* stackexchange.com

Thanks,
:-Dustin