The power of informal contracts

The power of informal contracts

The "Principle of Informal Contracts" allows anyone to create useful mashups.

by Jon Udell | @judell | Comment | 11 August 2010

Before releasing elmcity as an open source project, it had to be reviewed by a team of Microsoft lawyers. They found no patentable invention in the code and gave me the green light. Which is funny in a way, because I'm sure I couldn't ever create a patentable software invention. I'm just not that talented as a programmer. Writing code, for me, is mainly a way to explore ideas and illustrate possibilities. In this case, it's the idea that we can manage data in a social way by participating in networks of syndicated feeds. And it's the possibility that a lone developer, modestly capable but empowered by languages, libraries, tools, and cloud services, can bring that idea to life.

What is innovative about my project, I claim, is the network-oriented way of thinking and acting that it embodies and promotes. The approach is based on a set of principles that we have yet to fully articulate, never mind teach along with the proverbial three R's. Almost everybody learns the rules of reading, writing, and arithmetic. But almost nobody learns the laws that govern the structure and flow of information in networks, and that enable us to make effective use of those networks.

We don't need software innovation to solve the problem that the elmcity project addresses. Our perpetual inability to merge personal and public calendar data can't be blamed on a lack of standard, broadly-available software and services. We have all the stuff we need, and have had for quite some time, and it all interoperates pretty well. But we haven't internalized why, and how, to use it in a network-oriented way.

One of the crucial ingredients is something that I'll call the Principle of Informal Contracts. Here's an example from the early blogosphere: RSS auto-discovery. It's the mechanism that associates the pages of a website with a corresponding feed. Back in 2002 a remarkable collaboration brought it into existence, summarized here by Mark Pilgrim who concluded:

Thank you to everyone who has been working on making this come together in the past few days. It has been surprisingly painless and friction-free. Together, we have come up with a new standard that is useful, elegant, forward-thinking, and widely implemented. In 4 day.

Thanks to that consensus, it has ever since been easier to subscribe to blogs. But what about before then? The blogosphere's feed ecosystem was already bootstrapped and thriving. RSS, implemented by various feed publishers and feed readers, was the obvious enabler. But there was also an informal contract. It went something like this:

My blog publishes a feed at a URL that I will tell you about one way or another. Maybe I'll use an orange icon, maybe I'll use a subscribe link, maybe both, maybe something else. By whatever means, I promise to produce a flow of new items at that URL. The feed itself might be one or another flavor of RSS. Whichever it is, your feed reader can count on finding certain things in a predictable way: a title, a link, a description.

That was enough to get things started in a big way. Here's a more contemporary example: the contract that's implied by choosing a tag at the beginning of a conference. It says:

We, the organizers, promise to bind the online resources that we produce for this conference to this tag. And we invite you, the attendees, to do the same. We do not promise to collect all the resources discoverable by means of this tag, dump them into a bucket, and assert authority over (or assume liability for) the contents of the bucket. We just want to be able to find all the stuff, yours and ours, and use it as needed. We want you to be able to do the same. And we think the virtual network routed by that tag can be valuable to everybody.

Members of a certain tribe, which you most likely belong to if you're reading this, take that contract for granted. For you, the word "camp" does not connote outdoor recreation, but rather a new kind of conference that's co-created by the organizers and by the attendees. One of the organizers' roles is to declare a tag for the conference. I've watched newcomers to the tribe encounter this practice for the first time, and then immediately adopt it. So I know the idea is catching on.

But we haven't yet spelled out the underlying principle of informal contracts. And we've got to do that, because we're living in a world where networks of people and data can fruitfully use such contracts. They're easy to create if you know how and why. Here's the contract for the ecosystem of calendar feeds that I'm trying to bootstrap:

Anybody who wants to promote a series of public events agrees to publish a feed that transmits certain facts about events in a predictable way: the title, the starting date and time, a link back to the authoritative source. Anybody who wants to curate a view of sets of feeds made available in this way can do so by making a list of them.

Within the project itself, there are some other contracts. Here's one:

To make the lists of feeds that define these curated views, we'll agree to use the delicious social bookmarking service in a particular way. The elmcity service will define the extensible set of delicious accounts used for this purpose. We'll agree that users of those accounts will follow certain practices that define the settings for their instances of the service, and the lists of feeds they trust to flow through their hubs.

That agreement in turn enables another. Because every action in delicious is a database query that produces an RSS feed, the acquisition of a new calendar feed by a curator sends a message on a virtual channel that can be read and reacted to. The service that reads and reacts, in this case, is FriendFeed. Here's the contract:

Curators can subscribe to a project feed that's aggregated by an instance of the FriendFeed service. Items flowing through its Atom feed are significant events in the life of the project. They include:

The posting of a forum message or reply.

The posting elsewhere of a tweet, blog item, or other resource bound to the project tag.

The discovery of a new (and trusted) iCalendar resource by any hub's curator.

Like delicious, FriendFeed can support this kind of contract by empowering users to define sets of online resources and share them as feeds.

With these kinds of contracts in place, interesting possibilities arise. Here's one that tickles me: The elmcity service doesn't yet need to implement its own system of user registration. If my approach to decentralized event curation takes off, I might someday need to create a registration system and require curators to use it. But for now it's trivial for me to bookmark a delicious account and tag it as one that the service regards as authoritative for a hub.

Likewise, the folks who provide the feeds trusted by curators and aggregated by hubs don't yet have to register with the service. For now, providers need only make feeds known to curators, one way or another. And curators who deem those feeds worthy of inclusion need only bookmark and tag them.

These are the kinds of workflows that can arise when informal contracts are in place. They're cheap to create and easy to evolve. Here's a final example. If you're a curator who adds a new feed to a hub, you naturally want to see your new events merged into the hub as soon as possible. Normally you'd have to wait until the next aggregation cycle, which might be as long as eight hours. So I had to come up with a way for a curator to send an authenticated message to the service, telling it to short-circuit the wait and gather new events right away.

My first thought was that here, finally, was the reason I'd need to build my own user registration system and require curators to use it. My second thought was, hang on, I already trust their delicious accounts, why not use those accounts as channels for authenticated messages? That's doable, but I didn't find a graceful way to do it. My third thought was: What about Twitter? Hence the Twitter contract:

Among the settings that a curator conveys to the elmcity service, by way of a trusted delicious account, is the Twitter account to be used in connection with that curator's elmcity hub. The service will follow the curator at that Twitter account. By doing so, it enables the curator to send authentic messages to the service. The vocabulary used by those messages will initially be just a single verb: start. When the service receives the start message from a hub, it will reaggregate the hub's list of feeds.

In a companion how-to article on O'Reilly Answers, I show how this piece of the service works. What's relevant here is that the code doesn't have to do very much. That's because the informal contract makes it possible to reuse Twitter in a novel way, as a channel for authenticated messages.

In a world full of services like delicious, FriendFeed, and Twitter -- services that can route feeds of data based on user-defined vocabularies -- you don't have to be a programmer to create useful mashups. You just have to understand, and find ways to apply, the Principle of Informal Contracts.