on News

Google Joins the Cloud

“Cloud computing” is the new buzz term in programming circles these days. It describes a kind of grid computing. From Wikipedia (paraphrased):

In general, the label suggests that function comes from "the cloud" -- often understood to mean a public network, usually assumed to be the Internet -- rather than from a specific identifiable device. While most traditional IT clouds are made up of heterogeneous systems, an ensemble is a pool of homogenous systems within the cloud which are compatible with one another. These ensembles are integrated by virtualization and management software which allow for mobility of the software stack between physical servers. It can scale from a few servers to many thousands, while having the management complexity near that of a single system. For instance an ensemble could be a set of systems which are capable of running a certain web appliance stack. The operating system, middleware, and web-app run exactly the same between servers in the ensemble.
Basically, you have online computation or storage functions from a computing grid made up of tons of devices. From a developer's standpoint, you write like it's one device; in the compute cloud, it will scale to use as much hardware as needs (or your checkbook will cover) from a cloud shared by lots of other devs.

This is fundamentally different from virtualized servers or clustered servers or software as a service. It’s a brand spanking new way of doing things, and it changes the nature of IT. You get virtually unlimited scalability (horizontal or vertical), 99.99% uptime (theoretically), and you can get rid of your power-sucking server closet and the hardware maintenance that goes along with it.

Amazon is the big player in this space, with their Elastic Compute Cloud (EC2) and Simple Storage Service (S3). With S3 you can store data of any type, with a max file size of 5GB. Files are indexed for fast access (SimpleDB, based on Erlang), and massive parallelism makes speed and uptime top notch. With EC2, you’re essentially building a “cloud” machine, with multiple operating system options. For example, if you wanted to put your web/database server in the cloud, you could create a Fedora Core instance with MySQL and Apache running your site. The instance gets stored in S3 and run by EC2. Fees are based on things like storage size, bandwidth, etc. Even though it’s still in beta it’s very popular - you can see some success stories here. There are already some GIS apps taking advantage of S3 - Tilecache can access tiles stored there.

If your server goes down now, you will probably spend a day getting new hardware up and running, with your customer spouting various derivatives of “you suck” every minute of the way. With EC2, you can launch new servers in an instant. Need redundancy? Replicate your database on 10 servers. Your Apache instance is dragging? Fire up 10 more servers just for caching.

Not wanting to be left out in the cold, Google has joined the cloud computing space this week. There offering is quite a bit different from Amazon’s.

Google App Engine is basically that - an application engine. It is “an application-hosting tool that developers can use to build scalable web apps on top of Google’s infrastructure.” While with EC2 you run machine instances with any number of different tool sets, Google App Engine is a one-stop shop.

You can get a better idea of how it works from this Video:

Some notes about Google App Engine:

  • Your application runs in a sandbox - a secure environment just for it.
  • Your traditional RDBMS is ditched in favor of the recently popular properties type interface (think SimpleDB).
  • Right now you can only program in Python, but they are talking about more languages down the road.
  • Python frameworks (Django, Pylons, anything that supports CGI) are supported.
  • The developer SDK is available for Windows, Mac, and (drum roll please) Linux. Woot! Python 2.5 is required.
  • Google App Engine is only in limited release to the first 10,000 devs that sign up, and if you haven't signed up already, forget about it. They'll open up after they kick the tires for a bit.
  • The amazing news is the cost - for 500MB of storage and and up to 5 million pages views a month, it's free. As in free beer.
Google App Engine is brand new, so we'll have to see where it goes. But one thing it may do is drive down the cost of competing services like Amazon's.

This is an exciting time to be an application developer. Cloud computing offers developers a lot more control and opportunities than they have ever had, and costs are becoming more and more negligible.

To this point my only work in the clouds has been Rorschach related daydreaming, but I hope to change that soon. As soon as I can see over the pile on my desk….