How do we get government to share data?

A look at three approaches to opening government data streams.

On Tuesday we wrapped up the Manor Makeover in Manor, Texas, population 6,500. In some ways, this is ground zero for Gov 2.0 at the local level. The City of Manor has done some very innovative things on a shoestring, gaining attention ranging from the blogosphere to national press and all the way up to the White House. In fact, keynote speaker Beth Noveck, Deputy CTO in the Office of Science and Technology Policy at the White House wrote up a blog post just last night. The makeover is pretty impressive — they even followed my blog post from earlier this year about how to embrace Gov 2.0. (Not that they’ve seen it — it’s probably just obvious if you think hard enough about prioritizing limited resources.)

The champions of the Manor event (which “made over” a different city, De Leon, Texas) are Dustin Haisler (Manor CIO) and Luke Fretwell (GovFresh). They are both well ahead of the open government curve, and I think they’re absolutely on the mark: with a little elbow grease, the tools exist to make government more collaborative, transparent and participatory on a limited budget. With the right political backing, connections, knowledge and personal drive, a lot can be accomplished toward making government act more like a platform, as Tim O’Reilly and others have envisioned.

But how do we get this “perfect storm” of talent and resources to see it take off? There are more than 18,000 cities and counties in the United States. Are any others doing it this well? Actually, yes. But I can pretty much count them on one hand. Watching (and giving) some of the Manor presentations, I participated with the group in the promise of government as a platform. Bonded by our collective vision, we all bring something to the table: code, software, political/administrative expertise, or perhaps just a dose of unbridled enthusiasm. However, something is “rotten in the state of Denmark.” Just yesterday, Luke posted a blog entry announcing the “end of a GovFresh era” — and this, one day after its most successful event.

As I listened, my mind kept wandering back to one common showstopper: without laws to compel the sharing of local data, it’s never going to happen at scale the way people are hoping. Even if we can get past the biggest issues — inertia and resistance to change — municipalities still don’t have the budgets and technical know-how to make it happen. Oh I know, I know — there’s always the argument that “the ROI on giving out data is a no brainer”…sorry, but the evidence isn’t there yet. If it were so compelling, more governments would be doing it.

How, then, does the public get access to data, and ideally, to raw data streams? The typical hurdles to sharing information — technology, budgets, politics, and the absence of laws requiring its disclosure — are particularly acute in law enforcement, so it makes a good case study. Tight budgets are forcing many agencies to cut personnel — and budgets were already tight before the recession. Not to mention that crime data is at best controversial and at worst vulnerable to statistical manipulation by whomever can get their hands on it. So what can we do? As far as I can tell, there are three basic tactics:

  1. Using force, by changing the laws or creating new regulations requiring agencies to disclose data and provide it as a machine-readable feed. (Note: some sunshine laws give access to limited data, but not as an ongoing feed or through an API.) This will take forever, so it’s not a great option.
  2. Using intimidation, by enlisting the news media to pressure the agency, or by hiring lawyers to threaten them with lawsuits, which have no basis in law or fact (caveat: I am not a lawyer, but I have hired some to help me be more informed). Although some agencies might capitulate, most won’t. And then they’ll be put on the defensive.
  3. By creating value for agencies to entice them to share the data. Sometimes it’s as simple as asking; other times, they need to see how they will benefit or they won’t share anything. Still other times, it’s a non-starter.

Let’s talk about crime data

So what, exactly, is public data? Is crime data actually public data, and do agencies have to provide it? If so, why don’t they provide it as a raw data feed? The answer is this: while crime data concerns the general public, it is not exactly public data per se, and individual agencies have their own rules. Most won’t — or at least don’t — release it. And I don’t know of any agency, anywhere in the country of any size, that feels it is legally obligated to disclose any crime data to anyone as a machine-readable feed. Period, end of story. Please, prove me wrong.

So, given that #1 and #2 above are not going to be effective, and that there are no requirements to disclose data in an open format, let’s assume we have to use option #3 above, and create value to entice agencies to participate. We still have to deal with the barriers: budgets, technology and politics (fear, apprehension and skepticism about what will happen if they share it). While the barriers may not be completely justified, they are real.

The CrimeReports approach

At CrimeReports, there’s no question about it: we work for law enforcement. We create solutions for them. Because the laws don’t compel them to share information at all, they hold all the cards, so we draw them into voluntarily sharing crime data by showing them how it creates value for their agency. We solve their technology hurdles, and we make it affordable. Let’s face it, they have other things to worry about, like fighting crime.

One challenge on “making it affordable” is that the sweet spot price is just $100/month on average, and some still can’t afford it. No surprise that at that price, we lose money, and a lot of it, since it’s not easy to extract the data, standardize it and perform all of the work to make it available and easy to use. If it were easy, believe me, it would already be done.

The bottom line is that I think it’s good to have incident-level data available to the public in near real time. It’s good for communities. It’s good for the agency that provides it. And it’s good for individuals. But we need a business model that works. If we want that data to be out in any form, we need to have the ability to make up for the loss of our cost on performing that service.

To keep us in check, the invisible hand will exercise its magic. If we price it too high, someone else can move in and replace us, which has already happened (we actually have other viable competitors in the space). If you don’t think that will happen, think again. Our contracts are cancelable with 30 days notice, and we don’t have the exclusive ability to source the data. If you, your company, the Department of Justice, or the local newspaper wants the raw data, nothing is preventing that from happening. (In fact, some of our customers publish crime data to more than one entity.) You just have to make it worthwhile to the agency, since the other two options listed up above are unrealistic.

In a perfect world, every government agency in the country would provide an open and free data catalog. The early adopters have done it. More will continue to do so. We’ve come up with a process that works: it gets data out into the hands of the public, and that’s a step in the right direction. So let’s not let the perfect be the enemy of the good.

The discussion is far from over, and I think there’s more we can do. I’d love to hear what you think and keep the conversation running.

tags: , ,