Resolving the contradictions between web services, clouds, and open source

Part 1 of the series, "What are the chances for a free software cloud?"

Predicting trends in computer technology is an easy way to get into trouble, but two developments have been hyped so much over the past decade that there’s little risk in jumping on their bandwagons: free software and cloud computing. What’s odd is that both are so beloved of crystal-gazers, because on the surface they seem

The first trend promises freedom, the second convenience. Both freedom and convenience inspire people to adopt new technology, so I believe the two trends will eventually coexist and happily lend power to each other. But first, the proponents of each trend will have to get jazzed up about why the other trend is so compelling.

Freedom is promised by the free and open source software movement. Its
foundation is the principle of radical sharing: the knowledge one
produces should be offered to others. Starting with a few
break-through technologies that surprised outsiders by coming to
dominate their industries–the GNU C compiler, the Linux kernel,
the Apache web server–free software has insinuated itself into
every computing niche.

The trend toward remote computing–web services and the vaguely
defined cloud computing–promises another appealing kind of
freedom: freedom from having to buy server hardware and set up
operations, freedom from installations and patches and upgrades,
freedom in general from administrative tasks. Of course, these
advantages are merely convenience, not the kind of freedom championed
by the free software movement.

Together with the mobile revolution (not just programs on cell phones,
but all kinds of sensors, cameras, robots, and specialized devices for
recording and transmitting information) free software and remote
computing are creating new environments for us to understand
information, ourselves, and each other.

The source of the tension

Remote computing, especially the layer most of us encounter as web
services, is offered on a take-it-or-leave-it basis. Don’t like
Facebook’s latest change to its privacy settings? (Or even where
it locates its search box?) Live with it or break your Facebook habit
cold turkey.

Free software, as we’ll see, was developed in resistance to such
autocratic software practices. And free software developers were among
the first to alert the public about the limitations of clouds and web
services. These developers–whose ideals are regularly challenged
by legal, social, and technological change–fear that remote
computing undermines the premises of free software. To understand the
tension, let’s contrast traditional mail delivery with a popular
online service such as
Gmail, a textbook example of a web
service familiar to many readers.

For years, mail was transmitted by free software. The most popular
mail server was Sendmail, which could stand with the examples I listed
at the beginning of this article as one of earliest examples of free
software in widespread use. Sendmail’s source code has been
endlessly examined, all too often for its many security flaws.

Lots of organizations still use free software mail servers, even
though in the commercial world, Microsoft’s closed-source
Exchange is the standard. But organizations are flocking now to Gmail,
which many people find the most appealing interface for email.

Not only is Gmail closed, but the service would remain closed even if
Google released all the source code. This is because nobody who uses
Gmail software actually loads it on their systems (except some
JavaScript that handles user interaction). We all simply fire up a
browser to send a message to code running on Google servers. And if
Google hypothetically released the source code and someone set up a
competing Gmail, that would be closed for the same reason. A web
service runs on a privately owned computer and therefore is always

So the cloud–however you define it–seems to render the notion of
software freedom meaningless. But things seem to get even worse. The
cloud takes the client/server paradigm to its limit. There is forever
an unbreachable control gap between those who provide the service and
those who sign up for it.

And this is apparently a step backward in computing history. Closed,
proprietary software erected a gateway between the all-powerful
software developers and the consumers of the software. Free software
broke the gate down by giving the consumers complete access to source
code and complete freedom to do what they wanted. Amateurs around the
world have grabbed the opportunity to learn programming techniques
from free software and to make it fit their whims and needs. Now, once
again, software hidden behind a server commands the user to relinquish
control–and as the popularity of Gmail and other services show,
users are all too ready to do it.

Cloud computing is leading to the bifurcation of computing into a
small number of developers with access to the full power and
flexibility that computers can offer, contrasted with a world full of
small devices offering no say in what the vendors choose for us to
run, a situation predicted in Jonathan Zittrain’s book

The Future of the Internet

Tim Berners-Lee, inventor of the World Wide Web, as part of a major
Scientific American article,

criticized social networks like Facebook
as silos that commit the
sin of hoarding data entered by visitors instead of exposing it openly
on the Internet. Ho, Sir Berners-Lee, that’s exactly why many visitors
use social networks: to share their personal thoughts and activities
with a limited set of friends or special-interest groups. Social
networks and their virtual walls therefore contribute to the potential
of the Internet as a place to form communities.

But Berners-Lee was airing his complaint as part of a larger point
about the value of providing data for new and unanticipated
applications, and his warning does raise the question of scale. If
Facebook-type networks became the default and people “lived” on them
all the time instead of the wider Web, opportunities for
interconnection and learning would diminish.

Complementary trends

But one would be jumping to conclusions to assume that cloud computing
is inimical to free software. Google is one of the world’s great
consumers of free software, and a supporter as well. Google runs its
servers on Linux, and has placed it at the core of its fast-growing
Android mobile phone system. Furthermore, Google submits enhancements
to free software projects, releases many of its peripheral
technologies as open source, and runs projects such as
Summer of Code to develop
new free software programs and free software programmers in tandem.

This is the trend throughout computing. Large organizations with banks
of servers tend to run free software on them. The tools with which
they program and administer the servers are also free.

A “free software cloud” may seem to be an oxymoron, like
“non-combat troops.” But I believe that free software and
remote computing were made for each other; their future lies together
and the sooner they converge, the faster they will evolve and gain
adoption. In fact, I believe a free software cloud–much more
than the “open cloud” that
many organizations are working on–lies
in our future. This series will explore the traits of each trend and
show why they are meant to join hands.

Next section:

Defining clouds, web services, and other remote computing

tags: , , , , ,
  • Andy,
    great piece, but I disagree on one (not central) point:

    “Free software broke the gate down by giving the consumers complete access to source code and complete freedom to do what they wanted”

    Consumers (or average humans, or end users, if you prefer) NEVER wanted access to source code and the freedom that comes with it. They couldn’t care less. Programmers care about that. In my opinion, giving for granted that consumers-or-users = programmers is a dangerous mistake

    I’ve already explained this in detail in the page below one year ago (look for the part starting with “We aren’t in the ’80s anymore”) so I won’t elaborate on it further.


  • ap

    I think the answer is P2P or otherwise completely distributed software. The bittorrent network is open source and open everything. Users are hosting it because the client is also a server. We could even have a bittorrent for storage and for computing, a P2P cloud. We are not there yet but look at volunteer projects like Boinc and Boss or software like Tahoe FS. People could share their unused computing power or share disk space as a form of redundant storage. The cloud is antithetic to the open source because it it’s a new take on centralized computing and open source can’t support that financially, let alone culturally. There is a lot of technical challenges to get there but I think it is imperative for the Open Source movements to make a complete shift to P2P, fully distributed solutions.

  • Alex Tolley

    OTOH, free, open source software opens itself to attacks. If the whole world uses open source software, we are just opening ourselves up to potential disaster.

    The cloud, as currently structured as a client server, is concentrating providers as single points of failure.

    What we need is more resilient systems. Most of the fuss over Facebook to date has been privacy and lack of user control over “their” data. What if that data was just corrupted or destroyed by malicious code?

  • I love these comments. They take off in many directions, and they’re
    all so pertinent.

    Marco poses one of the basic questions in the free software community:
    how can making software free benefit people who never look at the
    source code? I think, in the specific cases of clouds and web
    services, the last section of this article will propose an answer.

    Distributed solutions, security issues, availability–all those are
    topics in the third and fourth installments of my series, so I
    encourage the posters and others to come back and read them. I go over
    a litany of benefits and risks to clouds and web services (along with
    my opinions on them), but the list will be quite compressed and
    there’s plenty of room for disagreement or expansion.

  • Sam Penrose

    An entry-level smartphone has enough storage for all the words you’ll ever write. An entry-level computer has enough room for most of the photos and some of the video. In 5 years even the video will be a non-issue unless you are Justin Kan. If storage is separated from services — if instead of uploading your words and photos to FB, you allowed FB to point at them on a device that was always connected — the problem would be solved. The bandwidth, latency, and capacity are most of the way there. Something like Crashplan could be a CDN in reverse: you microcasting to the cloud instead of Bug Content broadcasting to you, decrypting only what you chose to share, to those with whom you chose to share it.

  • Mike

    I wish people would cease using the word “Cloud” or the phrase “the cloud” as it is NOTHING NEW. Its been around as long as networks have been around and all it is is some marketing buffoon’s new buzz term. Thats all it is.

  • James

    @Marco: How do you suggest software developers get paid? Of course Mr. OReilly doesnt mind this sort of thing… He has already “made it” so what does he care of the rest of us dont get to earn our millions. Sorry. Software should cost a pretty penny to LICENSE and use. Nothing less.

  • What keeps me worried these days about cloud-ish systems is not “free” as in “licenses” (the famous free beer) but gatekeepers, free as in free speech.

    Every once in a while, a wave of free-speech-questions washes over the internet. Right now it’s the question whether Amazon has the right to kick Wikileaks (AFAIK, they have) and whether it was right to do so (IMHO it was not). In France, there’s HADOPI (three-strikes for IP violators), in Germany there’s the discussion about the JMStV (protection of minors), in Canada, they are discussing whether links imply endorsement. All of these topics pop up in all of the countries at least once a year.

    At their heart, there is always the same question: Shall a private company decide what is and what is not permissible to publish?

    Because today, the internet may feel like a public space but it isn’t – it’s a space governed by corporations, by large corporations. Where government censorship was an issue centuries ago (or on other continents), now the challenge of corporate censorship is emerging right here, right now, often driven by government pressure.

    We are having the wrong debate, we are debating who may, under which circumstances, limit access to content. We should have the debate about how far we can stretch the right to access. Yes, there are important trade-offs to be taken. But today’s default behavior is in doubt to take down, and that’s leading us down the wrong path.

    I’m a technologist at heart, and the technical questions behind a free software cloud are fascinating. But at this point, I believe the question that has the biggest impact on our future are somewhere in the socio-legal-political environment. I’m also a utopist at heart.

    I’d like to see something like “In dubio pro reo” for content. “When in doubt, for the accused”. In doubt, content needs to stay available, and governments should force companies to keep it. Especially in those cases where today encourage pre-emptive removal.

    More in German at

  • Instead of expecting large companies to give away the data they pay a lot to collect, we should focus on the increasing accessibility of software tools made available for smaller developers to create their own clouds.

    Amazon EC2 makes it ridiculously easy to create a global server. Coupled with LAMP, the costs are very low and if you decide to provide a good API to other developers, you will have created your own open cloud on a budget. There would have to be a good commercial/philanthropic/technological incentive for this, of course.

  • 1. users don’t care about open source

    I think Marco Fioretti has hit on the essential point here: open source software is of great interest to programmers and of no interest at all to users.

    In the same way, when I buy a car I don’t care if I get a copy of the engine and wiring diagrams, since I’m not a mechanic, and I would never think to assert that cars would somehow improve in functionality or quality if the manufacturers all gave their intellectual property away for free — more likely the manufacturers would go out of business.

    2. for most people “free” means “I don’t have to pay”

    We will never have a really honest discussion of this topic when open source advocates persist in using the easily-misinterpretable word “free” when they mean “open source”. To use the car analogy, if all the manufacturers DID give us all their intellectual property and trade secrets . . . no one would ever say that manufacturers were providing “free cars”.

    For users, Gmail IS free in a sense that matters to users: it doesn’t cost anything to run and operate, and I can use it without having to pay programmers or IT support. Free = no money.

    [For the users of’s software, many of whom are in the poorer parts of the world, this type of free is especially important.]

    2. making everything open source isn’t viable economically

    Communism said “sharing is good, so let’s make EVERYTHING shared'” . . . and we all know how well that simplistic economic analysis played out in real life. Open source, like sharing, is also an excellent and important part of modern life. But since software developers need to pay for groceries and mortgages, and since it is harder to monetize something that can be taken for free open source always will and should be a PART of the software ecosystem, and not the whole.

    Saying “open source is good” is not the same thing as saying “Everything should be open source”.

  • I’m not sure I’m understanding how your final synthesis plays out. Google can use as much open source as it likes. That doesn’t make its web offerings any more free.

    What it could do is make it easy for users to extract the raw data (documents, emails, blog posts, comments, social network info, etc.) from its services so that they can be migrated to other services without too much pain. They’ve actually started doing this.

    But there isn’t anything (other than the ill will of consumers) stopping them from shutting the escape hatches they build into their own services, and no reason to expect competitors to follow suit.

    As to the idea of an “open source software cloud,” it’s not very clear what you mean by it or how such a thing resolves the dilemma. By the most obvious definition (a cloud running entirely on open source software) such a cloud would be no more likely to host “open” web services than a proprietary competitor.

    Perhaps a “GPL3” cloud might be useful: all services hosted on it must give users rights to the source code that runs the service, as well as the user data that makes the service valuable to its users. But I can’t imagine who would use such a thing.

  • We are in the attention economy. The focus of clouds should really be on OpenData, such as who owns the social graph, user-generated content, personae, etc. OpenSource has a role at the IaaS and perhaps Platform as a Service (eg witness all the hosted developement clouds springing up) but a pure service will require some form of differentiation which means some form of proprietary advantage (in google it’s their ranking algorithms).

    My basic premise is that users should own any generated data (eg location) and any social networking site or cloud should should hold that in trust for the benefit of that user (see Legally this is a constructive trust and not the current model where under guise of copyright, SNS grab all usage rights. Copyright is not well suited to collection of facts anyway.

    I think the general concern is the free-rider disquiet, that whilst some companies contribute back to OpenSource, others avoid their obligations. Now given the proprietary edge of SaaS it is unfair to insiste on banklet release but another angle is Corporate Social Responsibilkoity. Suggest any OpenCloud service to contribute XXX days of a year for non-charging model. This is both simpler than a tax and also you’d have a spectrum of services catering from fanatics who want something completely Afferro-GPL down to a custom host you you’d pay for but in knowledge that say 1-5% of time it is open to others to experiment with. Even the naked capitalist might be ostracised if out of an entire year living off the back of contributed code, they can’t even give back 1 days usage.