I have been thinking quite a bit lately about the new relationship that we, the computing community), have to power. I don't mean <I>power</I> like the ability to bomb third-world nations, but rather <I>power</I> like the grid that delivers electricity to your home and office.
In this day an age, I don't think anyone really wants to generate their own power. There is that really funny sportscenter ad with Lance Armstrong on a bike, but I think the closest most people want to get is solar panels on their roof and maybe occasionally checking the meter. They sure don't want to be dealing with the distribution grid, failovers, safety, and spot auction pricing.
<object width="425" height="355"><param name="movie" value="http://www.youtube.com/v/2XHl-WrefNE&rel=1"></param><param name="wmode" value="transparent"></param><embed src="http://www.youtube.com/v/2XHl-WrefNE&rel=1" type="application/x-shockwave-flash" wmode="transparent" width="425" height="355"></embed></object>
Why do people not want to generate power? Well, for one thing it's a big hassle to do safely and it scales well in terms of cost and transmission. Admittedly, Thomas Edison wanted us to use DC power and have a generating plant in every little community, but these days we'd rather have it done centrally (by professionals with the right training) and have it distributed to us. We use metering to keep people from wasting and, in some cases, to make a small profit from the enterprise. Can you imagine the computing world if we didn't have standard power generation and distribution?
Intern Job Opportunity: Mill wheel slave at Transmutable networks! You and your friends can keep our servers running!
In manufacturing, there is the notion of a value-chain. I'd argue that bending the strict definition is ok when thinking about how a customer gets an internet service delivered. At the front end of that chain are people getting paid for generating electrical power and getting to your domain's servers; one would hope they get paid a bit more than the amount it cost to make and deliver the juice or that they at least feel satisfied by providing a public service. On the other "end" of the internet service are the bit providers that actually route packets between your customers and your systems. Again, these folks get paid for doing something that most people would rather not do themselves. Slightly closer to the middle of this mental image of a value chain, but next to network pipe, might be other network services that one pays for like name service, content caching, etc. Again, these are things best done by professionals with special tools and training, not the typical web developer.
There is, though, a big gap in the middle. A typical web N.0 service hopes that customers come to it's web site and are willing to trade money (or something believed by somebody to equate to it) for whatever the service provides. To my way of thinking, that is bottom line for building a web service. The interesting thing is that, at least for most people, there are a bunch of software layers involved that add value that the developers are currently required to pay for either in cash or time. Simple examples here are operating systems, database servers, web development tools, graphics editing tools, compilers, etc. Sure, all of these are valuable and necessary, no doubt, but why can't we apply the 'I don't want to generate power' argument to all of them? Why are they not scaled up and delivered centrally, metered to make sure people get paid, and generally taking out of the view of the service developer?
We are part of the way there. Amazon's Elastic computing cloud ("EC2") is a good start. (If you don't know you better ask somebody!) EC2 allows you to rent computing resources of various sizes and pay for them by the minute. When you want to do something that needs computing you fire up an instance of EC2 and tell it what "AMI" to boot. An AMI is snapshot of an operating system on disk, so when you boot the image it starts up the OS and runs the usual initialization scripts. The person who booted the instance can access the instance via SSH or whatever other services (such as the web) are desired. Amazon provides you with tools to take an existing AMI and modify it, creating your own custom image. In simplest terms, your custom image boots up a web server, a database server, installs your application and away you go as a web service!
EC2 removes a bunch of the layers in the (or lack of a) value chain on the side near the electrical outlet. True, there were co-location and hosting companies before, but amazon steps it up a level from just the physical part--maintaining power, cooling, monitoring--to include the delivery of operating systems and some built in software. (This certainly is going to be a challenge for the licensing model of Vista since Amazon can make a great deal of free, open-source software available to developers with this model.) As a very satisfied user and paying customer of EC2, I feel that I can criticize it though for not going far enough.
Amazon has two great features in addition to being able to rent computing by the minute. One is the ability to have simple, long term, managed storage via Simple Storage Service (S3). Why in the world do I want to mess with "value chain" of folks who try to provide backup services when amazon is already had to develop all the infrastructure to do this for their own use anyway? I'd rather focus on my application, thank you. S3, however, is probably one order of magnitude slower than the local disk on an instance, so this is a good place to store big things that you want to be sure can be accessed, but not for something that is hit often. (Of course Amazon EC2 AMIs are themselves stored in S3!) For something more like a database that you are going to use often, they have the Simple DB. It's pretty much a quick hashtable that they manage for you. These pretty much cover the needs of most developers on the storage side since there is a fast, small one and a big, slow one.
I guess you knew that given my introduction in terms of electric power and 19th century, I was going to take this all the way. Why stop at these services and offer them as separate, non-integrated pieces? (There are a number of other cool services they offer that I didn't mention.) Trevor and I have been kicking around "the War Room" as the endgame of this process for the Java community that is developing web services. (In fairness, the rails world has got a pretty good stab at this via RightScale.) Our idea for the War Room is that everything that isn't your application's functionality is handled like electrical power: somebody else does the nasty work and you pay for what you use. This includes fixed infrastructure internet services like DNS and email, app servers like Tomcat, as well as simple mappings to service objects as Plain Old Java Objects (POJOs) that correspond to things you will need, such as the simple database or payment services. Basically, you would just upload a java-based WAR file that meets our specification--quite different than the existing deeply complex specifications--and it gets installed in your domain as your application. All the infrastructure is provided by the various amazon services and linked to java objects that you can use in your application. You still have to supply the big idea, though!
Critical to the idea is that the WARs that the War Room supports are pretty simple and known to have a great many properties--for example that the web APIs are REST and do not have state associated with each user's web session. (If you want the full flexibility of servlets and the like, use a generic AMI and feel free to do it yourself!) This allows the automatic use of load-balancing DNS services. As the web service grows in terms of number of concurrent users, app servers (instances) can be added to the application server tier as needed. This process, bringing up app servers loaded with your software, currently can be done pretty easily on the granularity of 4-5 minutes. If higher response rates to user load are needed for your application, it would require "over provisioning" the service and could be done if needed. Since you get billed only for what you use, you can choose if you want to over-provision or not.
I really think that the War Room could simplify web service development substantially by allowing a developer focus on the key problems to their service--their particular mojo--without any distractions. It allows an easy way to meter and pay for all the other cruft the developer doesn't need to mess with and allows scalable, reliable systems to be built quickly. If only I didn't have a day job already....
Comments