Karmic Koalas Love Eucalyptus

Guest blogger Simon Wardley, a geneticist with a love of mathematics and a fascination for economics, is the Software Services Manager for Canonical, helping define future cloud computing strategies for Ubuntu. Simon is a passionate advocate and researcher in the fields of open source, commoditization, innovation, and cybernetics.

Mark Shuttleworth recently announced that the release of Ubuntu 9.10 will be code-named Karmic Koala. Whilst many of the developments around Ubuntu 9.10 are focused on the desktop, a significant effort is being made on the server release to bring Ubuntu into the cloud computing space. The cloud effort begins with 9.04 and the launch of a technology preview of Eucalyptus, an open sourced system for creating Amazon EC2-like clouds, on Ubuntu.

I thought I’d discuss some of the reasoning behind Ubuntu’s Cloud Computing strategy. Rather than just give a definition of cloud computing, I’ll start with a closer look at its underlying causes.

The computing stack is comprised of many layers, from the applications we write, to the platforms we develop in and the infrastructure we build upon. Some activities at various layers of this stack have become so ubiquitous and well defined that they are now suitable for service provision through volume operations. This has led to the growth of the ‘as a Service’ industries, with providers like Amazon EC2 and Force.com.

Information Technology’s shift from a product to a service-based economy brings with it both advantage and disruption. On the one hand, the shift offers numerous benefits including economies of scale (through volume operations), focus on core activities (outsourcing), acceleration in innovation (componentisation), and pay per use (utility charging). On the other hand, many concerns remain, some relating to the transitional nature of this shift (management, security and trust), while others pertain to the general outsourcing of any common activity (second sourcing options, competitive pricing pressures and lock-in). These concerns create significant adoption barriers for the cloud.

At Canonical, the company that sponsors and supports Ubuntu, we intend to provide our users with the ability to build their own clouds whilst promoting standards for the cloud computing space. We want to encourage the formation of competitive marketplaces for cloud services with users having choice, freedom, and portability between providers. In a nutshell, and with all due apologies to Isaac Asimov, our aim is to enable our users with ‘Three Rules Happy’ cloud computing. That is to say:

  • Rule 1: I want to run the service on my own infrastructure.
  • Rule 2: I want to easily migrate the service from my infrastructure to a cloud provider and vice versa with a few clicks of a button.
  • Rule 3: I want to easily migrate the service from one cloud provider to another with a few clicks of a button.

So how do we go about creating freedom and competitive marketplaces? The often cited ideal is the electricity industry because of the ease with which consumers can switch between utility providers. However, these providers all offer the same output; the electricity service is standardised. Furthermore we have no relationship to our utility electricity provider and we shouldn’t have to rewire our home just because we have switched.

In cloud computing we do have a relationship with our provider, in the form of our data, code, and meta-data. These will all have to be migrated if we switch between utility computing providers. By the same token we will need the output to be the same, so providers must offer the same interoperable service. The cloud computing industry must standardise the output of various layers of the computing stack for this to happen.

Naturally, more standards should emerge in the application space because there are more types of applications than infrastructure. However, if we want co-operation between providers then no single provider can ‘own’ any of these developing standards.

In the software world this means that emerging de facto standards are likely to be open source reference models i.e. running code. For a ubiquitous and well defined IT activity that is suitable for service provision, open source fundamentally makes more sense. Any value relates to the service provision rather than the marginal feature differentiation of a common activity.

Hence, open source, cloud computing, marketplaces, and standards will go hand in hand. In Ubuntu we will be launching a number of open sourced systems to enable users to build their own clouds. We’re going to be promoting open sourced reference models along with portability between service providers.

Our first steps in Karmic Koala will be at the infrastructure layer. We’re adopting the emerging standard of Amazon EC2 and providing our users with AMI’s and tools to run on Amazon EC2. We’re also providing users with an open source means of implementing their own in-house EC2-like cloud (Eucalyptus). This is not the end but the beginning of a journey, and we will be looking to build ecosystems at all the layers of the computing stack based upon portability and choice.

Mark made the comment that it would be apt to make it easier to navigate the ‘jungle’ of cloud computing. Navigation is inherently all about freedom of movement and freedom of choice. We intend to keep these ideals at the heart of cloud computing.

tags: , , ,
  • http://blogs.sun.com/eclectic/ Wayne Horkan

    Hi Simon,

    Nice piece, I’d been predicting the first set of standardised virtualised stacks, based around EC2 in it’s dominant position for a couple of months now, and it’s great to see you guys break cover and be the first to capitalise on this.

    http://blogs.sun.com/eclectic/entry/sun_microsystems_entrepreneur_country_cloud#20090209004

    Hopefully we can catch up at the next CloudCamp event (URL attached for those interested in attending).

    http://www.cloudcamp.com/?page_id=216

    All the best,

    Wayne

    P.S.

    @H.K. Re: “the friction as America transitions towards a service based economy (which it logically should be)”

    The US is already predominately a service based economy (only circa 19% of GDP is made up from contributions from manufacturing). I suspect that not all of it’s 302M population can all be in the service industry. Perhaps a more ‘balanced’ economy is required?

  • Bill OBrien

    So is Canonical planning on taking more of a lead in Eucalyptus and helping with the migration from academic project to genuine open-source project?

  • http://blog.gardeviance.org Simon Wardley

    Hi Wayne,

    I’m actually one of the MCs for Cloud Camp London, so I’ll be there. As for breaking cover, well it seems clear that the industry will have to quickly move towards standards (as open source reference models) in order to overcome adoption concerns.

    Hi Bill,

    We are certainly working with and helping the Eucalyptus team along this path.

  • http://blog.gardeviance.org Simon Wardley

    RE @H.K.’s comment : “the transition towards a service based economy”.

    There is an underlying transition occurring here which is common to all activities. This transition is known as commoditisation.

    In its simplest terms, commoditisation (as opposed to commodification) is a neologism which describes how a rare and poorly understood innovation becomes well defined and ubiquitous in the eyes of the consumer. In other words, it’s a transition that describes how a once rare, exciting and new activity (an innovation) becomes commonplace, bland and standardised (more of a commodity).

    The most often quoted example of commoditisation is the electricity industry and how this innovation led to the formation of national grids in the 1930s. Today, to most consumers, electricity is something you get from a plug and few companies describe their use of electricity as a source of innovation or a competitive advantage.

    The journey of an activity from innovation to commodity has numerous stages (for example bespoke, product and services) and each stage has its own forms of differential change (e.g. product innovation, service innovation). Some of these are incremental and some disruptive. However, the overall transition represents a continuous and sometimes hazardous change of an activity to greater ubiquity and better definition. There are numerous factors controlling the transition but for brevity I’ll simply state that not all activities will commoditise, some have physical, social and other constraints.

    For example the activity of CRM (an innovation in the 1980s when we didn’t even have a term for the name) has been through the various stages of bespoke (custom built solutions), products (including various product or feature innovations) and finally entered a service (utility provision) stage. There will no doubt be various waves of service innovation but the point to note is that the activity itself has become ubiquitous and almost feature complete. It’s more of a commodity and no-one can describe CRM itself as an innovation today.

    I mentioned this journey was hazardous because the shift of an activity from each stage (i.e. products to services) is highly disruptive for industries built on the previous stage. In I.T today, it is no coincidence that many of the hot terms have a strong service theme – service oriented architecture, web services, mashing up services and so on.

    Of course, this transition affects all business activities. Organisations themselves are simply a network of people (who come and go) and a mass of activities (which are in a constant state of transition with new innovative activities appearing all the time). Organisations only exist at the intersection of people and activities.

    This has profound implications for the way we manage activities because the tools, methodologies and techniques you use to manage an innovation (a dynamic class of problem) are vastly different from the tools, methodologies and techniques you use to manage a commodity (a static class of problem).

    This is why outsourcing by type of activity (i.e. marketing, I.T. or some other group) rarely works well and why blanket methodologies (i.e. we use Agile development or we use Six Sigma) can be counterproductive.

    Tools, methodologies and techniques need to fundamentally change with the lifecycle of an activity whether you’re talking about project methodologies or financial focus (worth, ROI or cost).

  • http://www.webappwednesday.com Michael R. Bernstein

    There is more than one level of infrastructure layering. While foundational projects like Eucalyptus are certainly important, I’m much more interested in open source projects a bit further up the stack, like AppScale:

    http://code.google.com/p/appscale/

  • http://friendfeed.com/webmaven Michael R. Bernstein

    So, what new stack-acronym will we get?

    How about LEAP (Linux, Eucalyptus, AppScale, Python)?

  • http://let.sysops.be/wiki/open_host Craig Hubley

    @HK, @Simon Wardley

    You’ve missed the most important aspect of moving to a service economy – commodification disappears. While we find it easy to express the idea of host services as a commodity because they will likely be more price-competitive and less likely to be able to “lock in” users with variant interfaces, it’s more likely they’ll compete more on ability to provide value-added services. What prevents most users of shared host services from moving now is not proprietary interfaces or configurations so bizarre they can’t be moved off without major work, but the daunting task of securing their own stacks 24×7 and dealing with minor OS and tools configuration issues on demand, which shared hosts tend to do better because they notice them before you do. Price of hosting is basically already not a factor to any serious online service and surely not for an intranet/VPN.

    So it’s not the commodity pricing or ability to move stacks around with the stack of a button that will be decisive but the ability to threaten to do so – to quickly abandon hosting services that offer inadequate web-spam protection, inadequate response to OS errors, and so on. Quick removal of unresponsive web hosts from the market could well mean an *end* to selling services at a loss and making up for it by treating users like dirt. Meaning we might see higher prices for uniformly better services. Which would be a good thing.

    Economically, consider also the long term trend to “de-materializing” products into service/warranty arrangements that let the manufacturer take care of installation, maintenance and eventual disposal – also known as “full lifecycle stewardship”. If you read http://natcap.org you will discover that a commodity economy is not what this looks like. If you read Pine and Gilmore’s “The Experience Economy” you will likewise discover that giving commodities and products away is part of the game if what you want to sell is higher valued service.

    Technically, I’d ask for something more like a chart showing what BSD jails, “Karmic Koala” and similar mechanisms do to remove host dependencies.

    Also read this http://let.sysops.be/wiki/open_host and feel free to edit it.