How to create sustainable open data projects with purpose

Tom Steinberg on making a website vs making a difference.

mySocietyThere has been much hand-wringing of late about whether the explosion of government-run app contests over the last couple of years has generated any real value for the public. With only one of the Apps for Democracy projects still running, it’s easy to see the entire movement being written off as an overly optimistic fad.

The organisation that I’m lucky enough to lead — mySociety — didn’t come from the world of app contests, but it does build the kind of open-source, open-data-grounded civic apps that such contests are suppose to produce. I believe that mySociety’s story shows that it’s possible to build meaningful, impactful civic and democratic web apps, to grow them to a scale where they’re unambiguously a good use of time and money, then sustain them for years at a time. Right now we’re launching a new site, FixMyTransport, that is trying to try to raise the bar for the ambition and scale of civic apps, so this seems a good moment to share some thoughts about what it takes to build good services and get them to last more than a few months.

You have to be just as focused on user needs as any company (and perhaps more so)

People have needs. Sometimes they need to eat, sometimes they need to sleep. And sometimes they need to send an urgent message to a local politician, or get a dangerous hanging branch cleared off of a road.

What people never, ever do is wake up thinking, “Today I need to do something civic,” or, “Today I will explore some interesting data via an attractive visualisation.” MySociety has always been unashamed about packaging civic services in a way that appeals directly to real people with real, everyday needs. I gleefully delete the two or three emails a year that land in our inbox suggesting that FixMyStreet should be renamed to FixOurStreet. No, dude, when I’m pissed it’s definitely my street, which is why people have borrowed the name around the world.

We learned this lesson most vividly from Pledgebank, a sputtering site with occasional amazing successes and lots and lots of “meh.” The reason it never took off was because, unlike the later (and brilliant) Kickstarter, we didn’t make it specific enough. We didn’t say “use this site to raise money for your first album,” or “use this site to organise a march.” We said it was a platform for “getting things done,” and the users walked away in confusion. That’s why our new site is called FixMyTransport, even though it’s actually the first instance of a general civic-problem-fixing platform that could handle nearly any kind of local campaigning.

Being focused on user needs means not starting things you think you probably can’t finish

In mySociety’s history we have run four calls for proposals, asking the whole world what we should build next. Like most idea gathering processes, there’s about 100 bad ideas for every good one, but the bad ideas have value in that they reveal a habitual digital era trait — being insanely optimistic about the effort required to build things to a high standard.

Now, clearly, I’m not saying it is impossible to hack brilliant things without piles of VC gold. But if you are going to hack something really, genuinely valuable in just a couple of weeks, and you want it to thrive and survive in the real Internet, you need to have an idea that is as simple as it is brilliant. Matthew Somerville’s accessible Traintimes fits into this category, as does FlyOnTime.us, E.ggtimer.com and doodle.ch. But ideas like this are super rare — they’re so simple and powerful that really polished sites can be built and sustained on volunteer-level time contributions. I salute the geniuses who gave us the four sites I just mentioned. They make me feel small and stupid.

If your civic hack idea is more complicated than this, then you should really go hunting for funding before you set about coding. Because the Internet is a savagely competitive place, and if your site isn’t pretty spanking, nobody is going to come except the robots and spammers.

To be clear — FixMyTransport is not an example of a super-simple genius idea. I wish it were. Rather it’s our response to the questions “What’s missing in the civic web?” and “What’s still too hard to get done online?” But we didn’t start building it until we knew we had the money, and we didn’t try to fit it into evenings and weekends. It was painful to wait and not rush with it, but it was the right thing to do to build something up to the expectations of an Internet-using public habituated to websites with billion-dollar budgets. And we are emotionally and financially prepared for the six months of rapid iteration that will follow once the public arrives.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code ORM30

Data is your servant, not your master

I love open data. I love structured data. I love data, full stop. But my love of data is not the same as respecting our users’ needs. There are more than 300,000 bus stops, train stations, ferry routes and so on in the FixMyTransport back end, munged together over months of hard work from dirty, dirty public data sources. Can you see any sign of this on the homepage? No sir, because users want to fix transport problems, not revel in our mastery of databases.

Demand fewer, larger grants from government and funders

MySociety got lucky. It was born into a period of high public spending, 2003/4, and its second ever grant was for 0.02% of a government funding pot worth more than a billion dollars — about a quarter of a million dollars. It was amazing luck for a small organisation with no track record, possible only because so much money was being thrown around. Those days are gone on both sides of the Pond, but governments everywhere should note that that funding of this scale got us right through our first couple of years, until sites like WriteToThem were mature and had proved their public value (and picked up an award or two).

In the subsequent few years, we saw the “thousand flowers bloom” mentality really take over the world of public-good digital funding, and we saw it go way beyond what was sensible.
Time and again, we’d see two good ideas get funding and eight bad ones at the same time because of the sense that it was necessary to spread the money around. It would be great if someone could make the case to public grant funders that good tech ideas — and the teams that can implement them — are vanishingly rare. There is nothing to be ashamed about dividing the pot up two or three ways if there are only a few ideas or proposals or hacks that justify the money. The larger amounts this would produce wouldn’t mean champagne parties for grantees, it would mean the best ideas surviving long enough to grow meaningful traffic and learn how to make money other ways.

After a long road supported by public grant funding, mySociety is now 50% commercially funded and 50% private-grant funded, but we’d never have arrived there without being 100% public-grant funded for the first couple of years. Now our key donors are philanthropic, with Indigo Trust in particular covering most of the core development cost for FixMyTransport.

Respect the geeks

All great technology projects have one or more über geeks at the heart of them. If you find the right über geeks, they’ll understand politics, society and users just as much as they understand their code. If you find someone as ferociously multi-talented as, say, Louise Crow, who built FixMyTransport almost single-handedly, listen to them and change your plans when they say “no.” Luckily, she said “yes” to building this project, and I hope those of you who care about civic tech give her the props appropriate to building something on this scale. Respect her, and respect the geeks like her, and you’ll be one step closer to civic app success.

Related:

tags: , , , ,

Get the O’Reilly Data Newsletter

Stay informed. Receive weekly insight from industry insiders.

  • steve white

    so only a handful of geniuses can do this work

  • http://www.vacationrentalnet.com/vacation-home-rentals.asp Joseph Adams

    Hey, you have made such an amazing job with MySociety! It’s great someone has finally understood people’s needs as they do.

  • http://blog.e-democracy.org Steven Clift

    As the likely original person suggesting FixMyStreet should be FixOurStreet to my friend Tom ;-), I enjoy his laser focus on the end
    user. I do want to counter and say that I do see a continuing problem with narrow, often one-time online experiences, from my community
    builder perspective. If “my” marketing and a focused problem-solving service brings more people into a service, how do you leverage that
    engagement as an organizer for broader civic purposes/good? What I really really like about FixMyTransport is that it really is that by
    definition it is a broader -OUR- public service. So “my” marketing aside, it will be interesting to see this effort unfolds.

    One quick lesson, our city-wide “online townhall model” – http://e-democracy.org/if – attracts some 1% of households for engagement that often lasts _years_. As Tom might say, most people
    don’t wake up and say, I want to engage my city government today. Too abstract. On the other hand, in a few neighborhoods in Minneapolis,
    our model using the exact same technology, rules, facilitation, etc. as a “neighbors forum” is attracting 20% household participation (or
    over 800 members in my neighborhood – http://e-democracy.org/se ). Someone loses a pet, joins us, then finds their pet … 9 times out of
    10 they stay on our general purpose neighbor-to-neighbor exchange and are exposed to the serendipity of local exchange they would not have
    click on or searched out. The silos of local interest can and must be broken at a level where the “common interest” in protecting or building
    your very local community trumps differences in ideology, ethnicity, income, etc. which often divide us.

    To me this points out the missed opportunity for mySociety and many other break through projects – you have place-based participants, why not put some trust into those who want to connect for more generalist local purposes and channel them into spaces so they can reach critical mass? I am continuously amazed by the capacity of local people, once they have an ice breaking online public space, to not just ask someone else to fix something for them, but to share ideas and take action.

    Steven Clift
    E-Democracy.org

  • http://blog.buzzdata.com Momoko Price

    This is a great post and I agree with a lot of it. But one of the things that I think is affecting the sustainability of open government data isn’t just how clever or streamlined the design of the apps are, but the disconnect and lack of visibility of the dialogue around data itself.

    Open data innovation tends to manifest itself in a sporadic, punctuated manner right now because its momentum comes from public campaigns, grants, and/or hackathons (which are still awesome, don’t get me wrong.) These are temporary innovation engines.

    What we need is a stable common carrier for discussing, editing, and improving data publicly and globally, not just locally or privately.

    A github for data, really.

  • Katherine

    Hi Tom,

    Blue sky question for you – what do you think a mashup of open data and community television would look like?

    Katherine