Everyone jumped on the app contest bandwagon. Now what?

The next wave of government app contests need to incorporate sustainability, community, and civic value.

The open government community has learned some hard lessons about app contests and sustainability, many of which, when expressed here on Radar last month, created a strong debate.

After reading Andy Oram’s post, Virginia-based civic developer Waldo Jacquith, who was recently recognized as a “champion of change by the White House for his work on Richmond Sunlight, published a sharp critique of government app contests in general. Jacquith writes that, after watching the applications presented at June’s Health Data Initiative Forum, he came to a realization that there are two categories of apps:

… those that were liable to exist six months later, and those that weren’t. These apps have got to have a business model, whether making money themselves, or being such clear grant-bait that it’s clear an organization will take them on to house. Otherwise it’s just a toy that will do nothing to benefit anybody. The exception is perhaps for government units that are not collectively persuaded that there’s value to opening up their data — perhaps such contests to put their data to work can serve as inspiration.

There isn’t an inherent problem in app contests, I don’t think,
but they’re probably not worth bothering with unless there’s a
simultaneous effort to foster a community around those data. There’s got to at least be a couple of ringers, folks with good ideas who are prepared to create something valuable. Otherwise I think app contests are liable to disappear as quickly as they appeared, a strange blip in the upward climb of open data technologies.

Volunteers work on projects at the second Crisis Camp Haiti at NPR headquarters in Washington, D.C.
Volunteers work on projects at the second Crisis Camp Haiti (January 2010) at NPR headquarters in Washington, D.C.

Clay Johnson, who gained significant experience guiding such
contests as the former director of Sunlight Labs, has been advocating that governments focus on building
communities, not app contests

“Whether it’s for procurement,
press, or community, the important part is that the app contest
deadline is the beginning of the engagement with the developers, not the end,” wrote
this summer.

Dan Melton, the chief technology officer at Code For America, described a
deeper issue for this “movement of makers” on the nonprofit’s blog. As entrepreneurs, civic hackers, open government advocates and urban leaders try to make government better, Melton highlights a key tension around scaling:

On one side, we’re trying to achieve policy change for
a more transparent, efficient and participatory government. On the
other, we’re making the tools and software necessary for that to
happen. We haven’t quite figured out how to meld the two movements’
successful organizing strategies.

In particular, Melton took a hard look at the return on investment
that the civic media community has received from app contests and
hackathons to date and found reason for both concern and hope:

Policy makers/political leaders champion city or social
contests, to which developers respond with dozens or even hundreds of submissions. So far so good. When the app contest is over, often too is the partnership. Maybe one or two apps will be adopted by the sponsoring entity; sometimes none. It’s very very rare that we see widespread replication or scaling of these efforts and applications across our movement. We could have an app contest in every one of 360ish metro regions, and not a single widely spread app as a result.

In fact, in the past year, I’ve counted nearly 80 hackathons, contests and other types of events in our space. At an average of 40 participants and say 10 hours (low), that’s 32,000 hours of cognitive surplus spent on software. This isn’t a problem of effort, excitement, time or energy. It’s a problem of scale, leveraging each other’s work and replication.

We make once, but we’re not very good at making many times. We
don’t lack from makers, just in our organization, 550 this year wanted to commit a year of their life to making. I’m excited about the opportunities for replication and scaling through CityCamp, Civic Commons, Code for
America, Open Plans and Sunlight Labs amongst others. Maybe it’s the engineer in me, but we’re really lacking tools for widespread engagement, coordination and replication.

If we’re a movement of makers, what do our factories look like?

Strata Conference New York covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Save 30% on registration with the code STN11RAD

Seeking sustainability

For people who have created or participated in hackathons, this may come as bitter medicine, but it’s in keeping with the agile
development culture that many coders now work within. If an approach isn’t working, analyze the problem, try a different solution, measure the outcomes, and learn from your failures.

One clear trend in government app contests in cities is a shift from “what’s possible with this dataset” to focusing
on the needs of citizens.

New York City Big AppsIn New York City, the NYC BigApps contest is trying to tie ideas to development. Chicago’s open government approach to an app contest, Apps for Metro Chicago, has focused explicitly on sustainability, requiring open source code, offering technical assistance and explicitly connecting communities with software developers.

There are other reasons to continue to refine the model that go
beyond connecting communities or app generation. As GIS developer Eric Wolf commented at Jacquith’s blog, prototyping and testing data are two valuable functions that government app contests can serve:

1. To test/validate the infrastructure used to “open” government data. App contests can provide an intense beta test by people who can provide precise feedback about what works and doesn’t work.

2. To demonstrate usability concepts around the information produced by a government agency. The app doesn’t have to live on, but the combination of information and interface the app created may guide future developments in the agency.

I would suspect that an app contest could become an important part
of standard government contracting. Or even contract validation. You want to build a system that streamlines an agency’s information use and makes it more transparent? Here’s a relatively cheap way to beta test the effort or even do some rapid prototyping.

Here are a couple data points for architects of app contests to consider:

  1. The winner of the first NYC BigApps contest is now a VC-funded startup, MyCityWay. While $5 million in funding isn’t a common outcome (in fact, it’s unique as far as I know) it shows what can happen when civic entrepreneurs decide to solve a problem for citizens that hasn’t been addressed in the market. In this case, MyCityWay offers a good digital
    city guide that’s populated with open government data.
  2. One of the winners of the second Apps for America contests is GovPulse.us. The civic
    behind the app, which provided a better way to browse the open data behind the Federal Register, the nation’s official publication for government rules, subsequently worked with government to redesign and relaunch FederalRegister.gov using
    open source and open standards. That outcome, available to all
    citizens to see and build upon was one of the best case studies for
    open government in 2010. In August 2011, Federal Register 2.0 launched an API, further moving to act as a platform built upon open source and open standards.

Lesson learned? Whether developers are asked to
participate in federal challenges or civic hackathons, it’s time for governments convening them to focus on sustainability.

There will be plenty of chances to apply that lesson in the months ahead. For instance, a California law hackathon planned for September 3-4 will offer an opportunity to explore an open source approach to CA law. That same weekend in Washington, citizens interested in greening the Internet by making Apps for the Environment are invited to an American
University hackathon
. And developers who want to pursue $100,000 in prizes in the Apps for Communities challenge are up against an August 31 deadline.

To its credit, the U.S. Environmental Protection Agency (EPA) has made a public effort to learn about the issues surrounding app contests. Last week, I moderated an EPA webinar on open data at the agency’s D.C. headquarters. The webinar featured a robust conversation about open data and app contests that touched on many of the critiques rendered above, along with persistent issues around government data quality, availability and structure. Jeremy Carbaugh of the Sunlight
and Michaela Hackner and Kurt Voelker of ForumOne shared their perspectives with me in the video embedded below:

The presentation used in the webinar is embedded below, including
useful links to resources.

Apps for the Environment
Developer Webinar

If you have feedback and ideas on how to make app contests and hackathons sustainable, let us know in the comments.

Photo: IMG_0597 by Divergence, on Flickr



tags: , , , ,
  • Canuck

    I used to talk to students about two broad reproduction strategies in nature: humans and great apes have very few offspring, but invest a huge amount of time and effort in caring for each one. Frogs have a huge number of offspring, but they have no involvement with them after birth, and most of them die before adulthood.

    The normal startup strategy is like the great apes, taking one idea, protecting it from harm, and nurturing it carefully

    These app contests, on the other hand, are producing a bucketful of minimal-effort tadpoles (low cost, high volume), so it’s not unreasonable to assume that most will die before they can mature.

    It’s not reasonable to expect much of a survival rate in this casual environment — in this kind of tadpole world, what kind of a success ratio do we need to make the contests worthwhile? 1 in 50? 1 in 100?

  • I suspect I’ve said enough about why communities ought to be the outcome of development contests, rather than the number of apps. What I haven’t talked a lot about is what to do after an apps contest finishes, and you’ve got your initial community. You can learn a lot about how to do this from the field operatives and people building political narratives.

    Developing communities like this involve three things: initial incentive, constant connectivity, and local discoverability.

    The apps contest is probably the best known model for providing the initial incentive for developers to start catalyzing around a movement and educating themselves about a certain set of data. But the contest’s incentive must be coupled with big picture stuff — fantasies of the kinds of problems developers can solve, and why they’re really important to solve. These ought to be tangible things that are plausible outcomes of any form of development around this data. It helps illustrate to a literal minded developer what kinds of possibilities are.

    That’s not to say that the apps contest is the only way you can do this. Sunlight, for instance, has a network of developers working on its 50 State Project that have come in largely independently from the Apps for America brand, and instead come from holding hackathons at O’Reilly conferences, and code sprints at other conferences due to the great organizing work of James Turk who doesn’t get nearly enough credit for the hard work he does there.

    The difference between the 50 state project and the apps contest model is really one of cost: Sunlight’s had to spend a lot more staff time on the 50 state project than it did on any of the Apps for America’s. Incentivizing with dollars can often be cheaper than incentivizing with time.

    After you have a community — you’ve got to find a place to put them, and usually a Google Group and a GitHub account will do. The challenge moves from acquiring developers and getting them excited about your issue to finding things for them to do and keeping them engaged. For many organizations, the whole point of this is so that they can have a community that can be a source of free labor. Here again, they run into a tempting trap. While that community may want to volunteer for the organization, if they implement a volunteer-like, top-down structure on their community they’re going to end up losing a lot of value.

    Instead, it’s important to understand that most of the ideas exist outside of the organization. In the case of Sunlight Labs, for instance, we’d never have thought to build a Federal Register 2.0, or build an amazing interface like Jeremy Ashkenas’s know-thy-congress.com. In addition, for Sunlight, because they’re largely grant funded they’re limited in the amount of non-direct funded work they can do. So the outside community actually has a lot more flexibility and creative power than the inside.

    So the role of a leader in this community needs to be, at this phase, keeping people inspired, connected to the organization, and driving connections to each other. I did this at Sunlight by bringing up, shortly after the second Apps for America contest the wiki bid on Recovery.gov, the 50 State Project, and the Great American Hackathon.

    Of the ideas at Sunlight that probably have the most potential for impacting these communities, I think the Great American Hackathon is the one that falls under the radar. There’s a lot to be learned there. Actually my wife’s idea, it was largely modeled after her work at MoveOn.org on the MoveOn Bake Sales of 2003/2004 and a way to discover local leaders. It’s largely how we found (and then hired) Jessy Cowan-Sharp who has been an incredibly talented organizer in the space.

    Once those folks are identified, it ought to be the role of the umbrella organization to build up those local leaders, provide for them a platform to build their own selves up (so they can be discovered) and drive people to them. That’s the last criteria: local discoverability. There are networks waiting to be tapped into, and probably thousands of developers who want to work on this but I think, especially where the national open government community falls down is in identifying and supporting local leaders, rather than competing with them.

    From an organizing standpoint, there’s still much to learn from political organizations. My colleague and friend from the Dean campaign, Michael Silberman said it best in noting the Dean campaign’s service oriented model: “The supporters dealt with us like they did with Maytag.” When building a community, the best thing you can do is provide them with big picture inspiration, identify the acolytes and evangelists for that vision, and connect them to the people around them.

  • What Clay said…

  • I think Clay has stated most of what needs to said. For Sunlight, Apps for America has been an incredibly useful tool for assembling a community of talented and civic-minded developers. We’ve got several members of staff who were introduced to the organization in whole or part through A4A (including myself); there’s the example of GovPulse/federalregister.gov; even some of the Chicago Tribune’s news apps team came together through A4A.

    To some extent I think that Apps for America plucked some low-hanging fruit, taking advantage of a backlog of developers who were looking to make a positive difference. There’s still plenty of potential there — the response that Code for America has received is proof enough of that. But many of the people who found this scene through A4A have begun working on problems that are too big for a contest. And with so many contests happening, it’s getting harder to attract a critical mass of talented participants.

    I think the slides from the EPA webinar — many of which were composed by Sunlight’s Jeremy Carbaugh –do a great job of explaining what’s required for success when running a contest. Much of the recent conversation about app contests has been about “sustainability,” but to me the real shift has been from making apps that demonstrate *potential* to ones that have genuine *meaning*.

  • Nice session Ethan and Alex! Sorry I missed it live – I really appreciate the work and efforts of all the folks featured in this session and the commenters :)

    We’d love to get some feedback regarding the recent HHS/CMS publication of Clinical Quality Linked Data on health.data.gov, of Todd Park’s Health Datapalooza this past 6/9.


    The above healthdata.gov blog post attempts to gently introduce the initial publication of Hospital Compare data as Linked Data, demonstrating how you can RESTfully dereference resources from datatypes and their instances to datasets and individual records in those, get any/all of that in a bunch of your favorite formats (JSON, Atom, RDF/XML, CSV, XTHML+RDFa, etc.), create/save queries through faceted browsing, do faceted search (based on the metadata ‘tags’ – except these tags have Web GUID’s), interact with other API’s and query points, etc.

    If you’re interested in how MS/Goog are amping up SEO, then you might also want to check out what an enhanced version of Apache Lucene/Solr can do with this HHS structured data – search on sindice.com for keyword ‘hospital’, then do it again using the http://health.data.gov/def/hospital/Hospital URI that defines this Class (a tag with a URI) to see a nicer looking, entirely data-driven UI with results that take you into another faceted browsing experience. We haven’t automated sindice.com indexing of our Hospital Compare Linked Data, I just pointed it to a few files, but you’ll get the idea.

    Glad to hear Jim H interacting with everyone – and also glad to see that people’s misconceptions and FUD about the ‘Semantic Web’ are eroding (like ‘one ring/ontology to rule them all’ – quite to the contrary, it drastically reduces the collaboration cost and easily intermingles data models from disparate sources!) as they witness more and more gov folks doing Linked Data (like NREL’s openei.org, HHS/CMS on health.data.gov, and soon EPA).

    It turns out that treating the Web like a distributed database really solves a lot of the problems and persistent themes being addressed not only by developers, but by mission owners, where lowering the cost of linking is inherently the most powerful thing – just as it always has been on the Web. So please don’t fear Linked Data folks – even if you’re still confused about the idea that Jim suggests, where Linked Data publishing standards and conventions presents a ‘normalized’, yes some might say standard API for all open gov data – one in which the same application code you wrote to visualize/process/persist all data from the last agency site gets reused on the next could be a tremendous boon to your productivity that allows you to focus on deriving or adding value through cross domain correlation – instead of everyone inventing yet another version of a simple CRUD API resulting in N-squared API integration, which externalizes the spaghetti architectures agencies have internally but feels more modern at the moment (at least it’s a start :).

    As far as outreach goes – all good points – and it would really be great to have more and more consistent interactions. I’ve been to #tcamp for the past couple years, and I try to make other unconf events whenever possible, but still, that’s not enough, and there’s not enough time in the days…

    I work with the Data.gov PMO with a bunch of gov Linked Data enthusiasts. We have regular meetings, where folks from agencies like HHS and EPA and others work toward mutual OGD goals and objectives. These meetings are open to anyone interested in joining in, so if you’ve got an hour or so every Wed, and you’re interested in Linked Data works in progress like I’ve described here, we’d be delighted to engage with you through this channel -just send me an email or dm or whatever – we’d love to hear more from you.



  • All great comments, and I agree that the focus should be two-fold: inspiration and engagement followed by sustainable business models.

    The first (inspiration and engagement) is necessary because you have to attract people to an initial community, or if you are building from a pre-existing communities, then you are likely wanting to bring in new members or innovators. Ideally, you have a challenge or contest that will connect two previously disconnected groups to lead to new insights.

    The second (sustainability) is critical based on all the great lessons folks have shared above and elsewhere online. At the end of a successful weekend of coding, the great feelings should be because something was started and will grow, not because an app was completed and archived. Having a community space for these groups is critical. These spaces exist in lots of places.

    On Data.gov we have them at Health.Data.gov, Energy.Data.gov, and Data.gov/opendata. When Todd Park stepped forth with the Health Data Initiative, he realized the problem with previous challenges and put the requirement for a sustainable business model for winners in 2011. He could point to the success of some of the winners of the 2010 challenge to show these start ups (or new apps for an existing company) examples of successful models. This is really the only way to be sure that all that great energy, innovation, and passion at a hack-a-thon/challenge/event can deliver real value over time to the community and citizens.

  • Scott Levy

    Fantastic post. Caspio is a great tool that lets you build any type of voting, contest, and other apps fast and without coding. This post showcases a sample app http://blog.caspio.com/web_apps/vote-for-the-oscars-contest-app-created-by-the-boston-globe/