Andy Oram

Why web services should be released as free software

Part 4 of the series, "What are the chances for a free software cloud?"

by @praxagora  | +Andy Oram  | Comments: 620 December 2010

Previous section: Why clouds and web services will continue to take over computing

Let's put together a pitch for cloud and web service providers. We have two hurdles to leap: one persuading them how they'll benefit by releasing the source code to their software, and one addressing their fear of releasing the source code. I'll handle both tasks in this section, which will then give us the foundation to look at a world of free clouds and web services.

Cloud and web service providers already love free software

Reasons for developing software as open source have been told and retold many times; popular treatments include Eric S. Raymond's essays in the collection The Cathedral and the Bazaar (which O'Reilly puts out in print), and Yochai Benkler's Wealth of Networks (available online as a PDF as well as the basis for a wiki, and published by Yale University Press). But cloud and web service companies don't have to be sold on free software--they use it all the time.

The cornucopia of tools and libraries produced by projects such as the open source Ruby on Rails make it the first stop on many services' search for software. Lots of them still code pages in other open source tools and languages such as PHP and jQuery. Cloud providers universally base their offerings on Linux, and many use open source tools to create their customers' virtual systems. (Amazon.com currently bases its cloud offerings on Xen, and KVM, heavily backed by Red Hat, is also a contender.) The best monitoring tools are also free software. In general, free software is sweeping through the cloud. (See also Open Source Projects for Cloud on Rise, According to Black Duck Software Analysis).

So cloud and web service providers live the benefits of free software every day. They know the value of communities who collaborate to improve and add new layers to software. They groove on the convenience of loading as much software they want on any systems without struggling with a license server. They take advantage of frequent releases with sparkling new features. And they know that there are thousands of programmers out in the field familiar with the software, so hiring is easier.

And they give back to open source communities too: they contribute money, developer time, and valuable information about performance issues and other real-life data about the operation of the software.

But what if we ask them to open their own code? We can suggest that they can have better software by letting their own clients--the best experts in how their software is used--try it out and look over the source for problems. Web service developers also realize that mash-ups and extensions are crucial in bringing them more traffic, so one can argue that opening their source code will make it easier for third-party developers to understand it and write to it.

Web and cloud services are always trying to hire top-notch programmers too, and it's a well-established phenomenon that releasing the source code to a popular product produces a cadre of experts out in the field. Many volunteers submit bug fixes and enhancements in order to prove their fitness for employment--and the vendors can pick up the best coders.

These arguments might not suffice to assail the ramparts of vendors' resistance. We really need to present a vision of open cloud computing and persuade vendors that their clients will be happier with services based on free software. But first we can dismantle some of the fear around making source code open.

No reason to fear opening the source code

Some cloud and web providers, even though they make heavy use of free software internally, may never have considered releasing their own code because they saw no advantages to it (there are certainly administrative and maintenance tasks associated with opening source code). Others are embarrassed about the poor structure and coding style of their fast-changing source code.

Popular methodologies for creating Web software can also raise a barrier to change. Companies have chosen over the past decade to feature small, tight-knit teams who communicate with each other and stakeholders informally and issue frequent software releases to try out in the field and then refine. Companies find this process more "agile" than the distributed, open-source practice of putting everything in writing online, drawing in as broad a range of contributors as possible, and encouraging experiments on the side. The agile process can produce impressive results quickly, but it places an enormous burden on a small group of people to understand what clients want and massage it into a working product.

We can't move cloud and SaaS sites to free software, in any case, till we address the fundamental fear some of these sites share with traditional proprietary software developers: that someone will take their code, improve it, and launch a competing service. Let's turn to that concern.

If a service releases its code under the GNU Affero General Public License, as mentioned in the previous section, anyone who improves it and runs a web site with the improved code is legally required to release their improvements. So we can chip away at the resistance with several arguments.

First, web services win over visitors through traits that are unrelated to the code they run. Traits that win repeat visits include:

  • Staying up (sounds so simple, but needs saying)
  • The network effects that come from people inviting their friends or going where the action is--effects that give the innovative vendor a first-mover advantage
  • A site's appearance and visual guides to navigation, which includes aspects that can be trademarked
  • Well-designed APIs that facilitate the third-party applications mentioned earlier

So the source code to SaaS software isn't as precious a secret as vendors might think. Anyway, software is more and more a commodity nowadays. That's why a mind-boggling variety of JavaScript frameworks, MVC platforms, and even whole new programming languages are being developed for the vendors' enjoyment. Scripting languages, powerful libraries, and other advances speed up the pace of development. Anyone who likes the look of a web service and wants to create a competitor can spin it up in record time for low cost.

Maybe we've softened up the vendors some. Now, on to the pinnacle of cloud computing--and the high point on which this article will end--a vision of the benefits a free cloud could offer to vendors, customers, and developers alike.

Next section: Reaching the pinnacle: truly open web services and clouds.

Comments: 6

Derek Balling [20 December 2010 12:11 PM]

I think comparing web services "use of open source tools and seeing the benefits" to "releasing their own web-service-platform source code" is comparing apples and oranges.

The Apache project doesn't compete directly with your typical web-services application. PHP developers aren't actively trying to steal users and mindshare from a given web site.

Contrarily, the people who would care about the source code to a given web-services platform are *very* interested in stealing market-share from the organization/company/etc. who released the source code.

It's a completely different set of questions, and while you mention a lot of differentiators, another huge differentiator in the web services industry is speed and efficiency. Google will be the first to tell you that how fast you can display a given page affects user-retention in a huge way. And speed, if your organization is smart, comes down to coding efficiencies. You need your pages to better written than your competitor's, so you can have that couple tenths of a second advantage over your competitor and retain marketshare. The last thing you want is to show your competitor the cool super-efficient thing you're doing under the hood so that they can start doing it to. It's just counterproductive to success.

Andy Oram [20 December 2010 01:34 PM]

Thanks, Derek, especially for your point that quality of service is a major aspect of winning the SaaS race.

As for the difference between using free software to develop a service and making your service free software: I haven't confused these personally, but my rhetoric might have left some confusion. I didn't have to say anything in the article about how cloud and SaaS companies use a lot of free software. But I've made a big deal about it (in two separate segments!). The reason is that I think it helps those of us who want to do what I recommend in this segment: persuade services to free their own software.

We don't have to tell the IT departments at these services what free and open source software is. (Isn't it amazing how many people in computing still misunderstand the basic concepts?) We don't have to tell the staff that free software can be high quality and can be respected by others as high quality. We don't have to tell them how free software communities operate and how to release software as open source. It's a big relief to pass these hurdles so quickly on this mission.

David Collier-Brown [20 December 2010 05:59 PM]

I think an important business reason to build a cloud on free software is to be able to have exactly the same software on an internal cloud, to handle the everyday load.

It's just a practicality argument: I want to use an internal cloud (a puff of smoke?) to run the everyday load, and then grow the service into a public cloud when I get hit by an increase in load.

If I use something proprietary or idiosyncratic, I probably can't grow into and out of outside vendors' clouds. I also have trouble finding developers and peers to bounce ideas and diagnostics off (;-))

--dave

Derek Balling [20 December 2010 06:16 PM]

David: I think that's a matter of "which proprietary cloud you choose". If you choose, say, VMware as your cloud-architecture of choice, you're going to find little to no problem finding developers and peers to bounce ideas and diagnostics off of. :-)

drllau [20 December 2010 11:13 PM]

Dave notes the difference between platform as a service (dev env) v software as a service (business logic for eCommerce or social media). The first is typically open but there appears resistence towards the latter. IMHO the difference is who pays for the NEGATIVE information. By negative information I refer to failed attempts and mistakes. Edison experimented on hundreds of filaments before settling on tungsten. If a competitor skipped all that and copied the idea directly, GE wouldn't exist today.

With drug design, the biopharm companies are big enough to swallow the failures as the big hits pay for the R&D of all the rest. In dev environments, it is instructive to learn from mistakes which is capitalised into the language design and libraries. With web sites ???? Here firms resort to trade secrets for profit maximisation and open sourcing it has a different economic calculus (eg IBM does it to flog their hardware and consulting services). Where the public good outweights private gains (eg voting software) then a strong case can be made for transparency but in general? Unless you address the free-rider issue, open clouds based on Affero+GPLv3 has the commercial attractiveness of a suicide pill (of course Netscape took that route after been steamrolled by IE but that was different circumstances).

I can think of a few circumstances where open clouds with all the code and logicl is freely available (eg repeatable scientific experiments) but in general, OpenSource business models are not that easy to execute but one possibility that occurs to me is to release code X years after first deployment (where X choosen according to standard industry practice). The innovator gets a small lead time (first mover advantage), it matches the general principle of industrial design rights, and hopefully avoids long-term rent-seeking behaviour (motivate continual innovation).

pc [21 December 2010 02:49 PM]

Thanks, Derek! Agree with software is more and more a commodity nowadays - but we need specialists to teach us how they works. In East Europe "learning on the job" in the common method to learn the new software.