Lessons learned building the elmcity service

Jon Udell's reflections on mashing up software cultures and calendar data.

by Jon Udell | @judell | Comment | 3 August 2010

In 1995 I started writing a column for BYTE about the development of the magazine's website, plus some early examples of what we now call web services and social media. When I started, I knew very little about Apache, Perl, and the Common Gateway Interface. But I was lucky to be able to learn by doing, by explaining what I learned to my readers, and by relaying what they were teaching me. Because I came to the project with a beginner's mind, the column became a launchpad for a lot of people who were just getting started on web development.

Nowadays I'm working on another web project, the elmcity calendar aggregator. And I came to this project with a different kind of beginner's mind. I had built a first version of the service a few years back in Python, on Linux, using the Django framework. After I joined Microsoft I decided to recreate it on Azure. I started in Python -- specifically, IronPython. But Azure was brand new at the time, and not very friendly to IronPython. So I switched to C# and .NET. I knew more about that environment than I had once known about Perl and CPAN, but not a whole lot more. That inexperience qualifies me to write another series of learning-by-doing essays, and that's what this will be.

The code, which is under an Apache 2.0 license, will live on github. I'll discuss it in detail over on O'Reilly Answers. In this space, I'll reflect on larger themes: building and operating a cloud service in 2010, in a way that cooperates with other services and straddles two different cultures.

You know the cultural stereotypes. In the open source realm, services written in dynamically-typed languages like Python and Ruby wrangle streams of open data for the public good. In the enterprise zone, services written in statically-typed languages like C# and Java manage proprietary data for profit. What happens when you mix open source goals, styles, and attitudes with Microsoft tools, languages, and frameworks? You get a cultural mashup. That's what the elmcity project is, and what this series will explore.

Recently I had dinner with Adrian Holovaty. He's the force behind Django, the popular Python-based web development framework, and EveryBlock, an engine for hyperlocal news and information. Adrian asked me what it's like to build software the way I've been doing it for the last year: in C# (and IronPython), on Azure, using Visual Studio Express. I picked the first example that came into my head: "When I rename a variable or method," I said, "it gets automatically renamed across the whole project." Adrian's response was: "I've never used a tool like that, so I don't know what I'm missing."

Of course it goes both ways. A lot of developers on the Microsoft side of the fence have never used Django, or Rails, and they don't know what they're missing either.

If you've followed my work over the years, you know I've always been a best-of-both-worlds pragmatist. So this will be an atypical narrative about C# and .NET development. I see through the lens of Perl, Python, HTTP, and REST, with a bias toward The Simplest Thing That Could Possibly Work.

You shouldn't have to drink a gallon of Kool-Aid, and then have a brain transplant, in order to start producing useful results. Back in the BYTE era I was struck by how little I actually had to learn about Perl and CGI in order to accomplish my goals. Likewise, I've barely scratched the surface of C#, .NET, Visual Studio, and Azure as I've developed the elmcity service.

I claim that's a good thing. There are many more services needing to be built than there are Adrian Holovatys available to build them. One of Microsoft's great strengths has always been the empowerment of the average developer. It should be possible for a useful service to be built, maintained, and evolved by somebody who isn't a great programmer. And trust me, I'm not. But the languages, tools, framework, and platform that I'm using for this project have enabled me to be better than I otherwise would be.

Finally, this series is about the wider goals of the elmcity project. It was born of my frustration with the web's longstanding failure to outperform posters on shop windows and kiosks as a source of information about goings-on in our cities, towns, and neighborhoods. I'm trying to bootstrap an ecosystem of iCalendar feeds that's analogous to the existing network of RSS and Atom feeds. The elmcity service is an example of what Rohit Khare memorably called syndication-oriented architecture. It embraces that style by syndicating with other services such as delicious and FriendFeed. And it will ultimately succeed only when everyone involved in the events ecosystem -- event owners and promoters as well as print and online aggregators -- can plug into a network of syndicated data feeds. So I'll talk about lessons learned while building and running the service, but also about why we need to broadly enable -- and popularize! -- a decentralized style of social information management. Because it's not just about events and calendars. We're all becoming publishers and consumers of many different kinds of data. Centralized repositories won't work. We have to learn how to network our data.