Thomas Lord [24 April 2009 09:28 AM]

Hooray for Aaron Swartz. To his observations we can add two more:

1. Power corrupts open government activists no less than anyone else.

With the arrival of the Obama administration the game changed for the "big names" in open government: suddenly, they had friends on the "inside" and a leadership (executive and legislative) that took up the rhetoric of open government. Suddenly they are "working closely with" a handful of government IT specialists and some legislative offices.

This is widely regard as a huge step forward for the cause yet it is anything but that. It's a disaster for the movement. It takes the pace and substance of the movement out of their hands and gives political ownership of the movements to the very politicians who should be under scrutiny.

At the same time the incoming administration and the Congressional leadership "took over" the project of even *defining* government transparency, the open government leadership was tossed some bones. They get to be "expert consultants" ("some of the brightest minds in industry", no doubt). They get some first-mover advantage setting up or extending existing their own transparency web properties.

The movement is at extreme risk of turning into something mostly about itself.

2. Journalism is only a small part of the answer. Organizing is needed, as well.

Aaron suggests investigative journalism as a way to cut through the politicization and agenda hoarding that threaten the open government movement. Certainly, it has a role.

What is palpably lacking in the open government movement is a broad base of support in the form of informed demand for transparency. As a society, we don't have much use for this data.

In the hands of demagogues (sometimes calling themselves journalists) isolated snippets of the data - nearly meaningless in their isolation - can be good for evoking and directing outrage at this or that.

What we lack, though, is any systematic habit of people self-organizing to examine the facts and then to act upon them in ways that meaningfully impact government.

In contrast, around here at least, people know darn well how to organize around the school system or the public transportation system. Citizen committees spring up like wild-flowers, around these things. Important hearings and elected official meetings are frequently well attended. What remains of the local press gives voice to these groups.

On federal issues? Or even state issues? Not so much. There is no impressive amount of local organizing at those levels.

Creating new "transparency web properties" is unlikely to change that. Such sites draw heavily on the technology architectures and business models of commercial web properties and in so doing adopt the form, hence the function, of an asymmetric power arrangement of powerful site owner / operators and a mostly passive consumer audience who, if their input is used for anything at all, it is to be aggregated to further enhance the power advantage of the site owner / operators.

It is as if, by shear professional habit, the open government movement made the two successive mistakes of (a) treating government transparency as a category of entertainment; (b) then not even being especially entertaining compared to, say, demagogue journalists.

A telling case in point is surely the recent "teabagging" protests. Boy, if ever there was a teaching opportunity to spread the power of transparency tools to grass roots organizers: that was it - yet the open government movement was nowhere apparent in the stories.

------------

My conclusion from these two problems is that while the open government movement surely must remain engaged with their friends in government that the course there is more or less set for the next few years and little will change it. Meanwhile, much progress could be made by changing the emphasis from building web site properties to building freely shared tools (IT and social) for meaningful citizen organizing and participation.

-t

Ken Williams [24 April 2009 01:52 PM]

Personally, I just can't get behind the cynical attitude in Aaron's column. There are probably areas where he's correct, and things are as terrible as he says, but there are whole other large areas where I think people by-and-large want to do the Right Thing.

Transparency isn't just important for cloak-and-dagger or loaded moral issues, it's also for mundane issues like letting citizens see where roads are planning to be built, or freely publishing the outcomes of research funded by government dollars, or seeing how budget allocations have changed over time. Taken all together, these mundane things can change the way people conceive of the interaction between themselves and their government, which IMO is pretty huge.

That said, shining light on something doesn't necessarily purify it. Many people throughout history have published meticulous records of their atrocious behavior for everyone to see. On the whole, however, I do believe it's a good thing, and helps lubricate the machinery of where the gears of the public & government come together.

Ajeet [24 April 2009 02:27 PM]

"To Except is Human, To Handle Is Divine" was too much of a pun for me to get right away.

@Ken Williams, I agree that the article seemed cynical. But one can get driven to cynicism when transparency turns opaque!

Michal Migurski [25 April 2009 01:13 AM]

Aaron's post is unfiltered cynicism: "more disclosure will only lead to more complicated lies." It is, as one of his commenters says, nihilism and hopelessness and for no good reason, so why is it being linked here? ABetterOakland.com's Tom Thurston recently wrote: "Policy is made by those who show up. Those who show up want something." A more interesting and productive way to look at transparency efforts is that complicated lying costs energy - more disclosure makes it more expensive, which is exactly the outcome we want.

Fortunately, it sounds like Aaron has decided to wash his hands of the matter because transparency "didn't work", like it was some product with a due date. Christ.

Thomas Lord [25 April 2009 10:36 AM]

Migurski,

Looking about at the ruins of the financial industry, would you say that predicting "more complicated lies" is cynical or realistic?

As you quote, "Policy is made by those who show up..." and that that's why the open government movement needs to turn attention to the problem of organizing and away from the never-ending, self-congratulatory project of making yet-another transparency demo site web property.

-t

Michal Migurski [25 April 2009 03:06 PM]

Thomas,

The exercise of producing transparency demos on the web is organizing. I don't know what you want to see happen, just like I don't know what Aaron wants to see happen - he's clearly frustrated, and not considering the situation in light of Micah & Clay's comment: no one ever seriously suggested that sticking a database online was going to fix everything, but it draws new lines between previously-disconnected actors and allows them to coordinate efforts: organizing. In your comment, you criticize the fact that many of the open government actors have found themselves on the inside, but I'm not sure what the problem is - maybe it's bad for the movement, but why should I care about a movement that decides to wallow in self pity soon as it gets close to a place where concrete goals can be acted upon with administrative authority? Screw 'em. ;)

I do see your point about the dead-end of government transparency looking like a form of entertainment surrounding the powerful site owner, but I just don't think it's all that relevant. Arthur Bentley: "The strength of the cause rests inevitably in the underlying group, and nowhere else. The group cannot be called into life by clamor. The clamor, instead, gets its significance only from the group. The leader gets his strength from the group. The group merely expresses itself through its leadership." Entertainment is clamor, noise is a proxy for support.

Thomas Lord [25 April 2009 06:42 PM]

Michal

By "organizing" I mean things like neighborhood caucusing and committee formation resulting in formal communications to government and journalists where those communications have intentional and effective impact. What I "want to see happen" is a shift towards emphasis on providing tools for *that* which can be used in privacy along with public outreach to spread the general idea and advertise what data resources there are. It's hard because there's a "wetspace" component to it that is far outside the boundaries of the comfort zone of the industry in-groups. But it is the only way forward and is consistent (indeed, essential, realistically speaking) to the ostensible goals of the open government movement.

-t

LoneKrazy [25 April 2009 10:59 PM]

hoooollyyyy... Be careful man... You are prety close to the definition of fascism... But in the other side... boring: making an article saying good and bad at the same time of the same thing...

Michal Migurski [26 April 2009 11:11 AM]

Thomas, I understand what you mean but I think you and Aaron are both letting the perfect be the enemy of the good here, specifically by letting aesthetic concerns around who is or isn't a roots organization obscure what I think are legitimate advances in government openness. People who know what they're talking about in open government have drawn close to the inside where they can have an effect - yay, what you describe is happening. Now we work to make sure they remain representative of what "openness" should mean.

Thomas Lord [26 April 2009 01:07 PM]

Michal, it begs the question to declare our critiques "aesthetic concerns" and brush them aside as a mere "good v. perfect" situation.

I think there is a serious conversation to be had there:

Aaron, it seems to me, observes that a false transparency is not necessarily a "good" threatened by a "perfect" but may instead be actively harmful. He is certainly correct for it is lately emerging that "government transparency" shall be defined and codified as rules, regulations, and recommendations. We can understand that as meaning that a bureaucrat or elected official challenged on the transparency of their office can point and say "we are fully compliant" and from that point the fight, if there is to be one, is a struggle over conformance to those rules and regs or struggle to modify them. Calling the emerging situation "transparency" is a mis-naming. It's a bit like calling government procurement process rules and regs "smart shopping". It's wishful thinking.

The essence of transparency in the ordinary sense of the word, before the opengov movement came along is that a thing is transparent if it puts up no barriers to inspection.

If I have a thing before me and I want to study its inner workings it is transparent if I may freely do so, but it is a black box if what is revealed for my study is limited and deliberately selected by the builders of the thing.

What we have in the current open government efforts is a power struggle over the size and form of limits to inspection of government. The fight is every bit as much about closing off parts of government from investigation as it is about rendering those parts of government that can be seen. It is in that sense not about transparency at all but rather about settling the rules of the games to be used going forward to limit inspection. The opengov movement is partially dismantling one black box in order to create a second generation black box. "Black box 2.0".

Suppose in a hard fought victory the open government movement frees up yet another database and, as is the custom, erects various web site properties to create interactive visualizations of the contents. In general terms, the net real effect within the bureaucracies is to perturb and re-shape internal practices to maintain tighter control over what information is added to the database, when, and by whom. Internal practices adjust to create operating room within the bureaucracy but out of site of the newly revealed database. And then things continue as before.

Aaron puts it well saying that transparency here is not the availability and slick presentation of that database but is rather the freedom and practice of investigators to explore every other aspect of the internal operation of the bureaucracy. His nomination of journalism is insightful: the journalist is in some sense a designated, public investigator. The journalists' presence and activity is necessary to empirically check whether a condition of transparency exists and to draw attention on where it does not. It is the journalists' job to recognize where the internal operations of government are most significant yet outside the scope of regulatory disclosure requirements and it is the test of government transparency the degree to which the journalist can shed light.

When we approach the goal of transparency by fighting for disclosure rules changes we merely warm up the internal game playing of government. The answers change, as the rules change, to questions like who is and who is not able to accomplish certain things in government but "out of the spotlight" - but the question doesn't go away and the capacity for black-box operation is not reduced (perhaps enhanced). It follows that of course such a movement would find dramatic support from within government - and especially support that initially flows more from one political party than another. And of course each party will eventually pander to such a movement because the when both parties are dissatisfied with and fighting over possession of salable opportunities to "quietly influence" government, both parties will be very happy to throw the rules of the game into a little bit of flux and then try to come out on top in the ensuing (mild) chaos.

And here is the kind of form and function we've come to: a relatively small group of tech elites have developed a lot of bureaucratic and political liaisons with government (and we note that many of the communications between these groups are kept very private). The main currency on the public side is web properties, board affiliations, press mentions, speaking engagements, symbolic executive positions, and so forth. The main currency on the government side appears to be struggles over regulation formation, interpretation, and implementation with a special eye on divisional budgets. For citizens we get some nice visualizations of data and the opportunity to click a "+1" to encourage the WH to begrudgingly answer a questions about marijuana legalization. When we see formations like this in countries we don't like we tend to call them something like an "a private propaganda engine adjunct to the government". I'm told the main reasons I shouldn't see that way here is that the players are "good people" and, anyway, my concerns are about aesthetics and I threaten to set the perfect against the good and, anyway, I can download the same files they are!.

Bah. I say, Bah. It's a fiasco. It's building displays of controlled revelations, not transparency. It's largely motivated by the relative personal power (political, economic, and social) of the opengov movement players.

And, no, I'm not all just criticism and would-be destruction - not all negative. There is a positive way forward and it's somewhere near what I've been talking about as "tools for organizing" and what Aaron is approaching in his take on "journalism":

How about "rights of interrogation and timely, substantial reply" as a good transparency issue? FOIA is a clumsy, blunt, slow, and largely ineffectual instrument and far to heavyweight for many situations. Perhaps instituting a system of ombudsmen to liason with investigators and a streamlined and low-fee court system to serve as first responders in disputes between investigators and government?

How about, as a technical contribution, something as simple as a "kit" for would-be organizers that helps someone in Median America set up their own little hub, print fliers, register where they can meet groups elsewhere who might share similar interests, prepare formal communications, make connections to the press, and initiate investigations of government for themselves? Not a "kit" with an ideological bent regarding any particular issue of the day: flat earthers and global climate change extremists both welcome - here's a set of very simple tools and general advice about organizing for each group. That kind of thing.

Please.

-t

Michal Migurski [27 April 2009 12:10 AM]

Thomas,

I think you and Aaron are both clouding the issue by reducing web-based transparency efforts to a list of hypothetical slick, black-boxed visualizations. Everything you're saying is true, but it's also completely uncontroversial. Obviously, the process of transparency is a negotiation: efforts to open up data from outside will be met with resistance, whether active cover-up, lack of understanding, or simple inertia. This is unremarkable, expected, and I think an unfortunate justification for the resignation you're expressing. Aaron's observation that false transparency is harmful is also uncontroversial and trivially true. I just happen to disagree that we're talking about "false" transparency here. =) Can you give examples, please, of the projects you think are actively harmful or the board affiliations you have a problem with?

Where I think you're both getting hung up on perfection is in your description of the Black Box 2.0. You describe transparency as a binary state, a thing being either open or closed, transparent or opaque, but I believe this is a mistake. For example, when we first built Oakland Crimespotting in 2007, we encountered a variety of impediments to our work: IP address blockades, slow responses from city hall, etc. The picture that ultimately emerged was simply that the officials involved are transitioning from a retail view of transparency (citizen writes a letter to inquire about a robbery, they mail back the incident report with personal information redacted) to a wholesale one (citizen inquires about every incident, every single day, expects RESTful XML). Although we came close to simply throwing up our hands, the OPD did eventually turn around and provide us with a reliable daily feed in spreadsheet form. It just took them awhile, government is accustomed to longer timescales and more conservative motions. Generally speaking, the kinds of people who go into government IT go in for the long haul, and they can be excused for not having gotten the memo about Web 2.0 three years ago.

I'm not happy with the nomination of journalism as a solution to the problem, either. You may have heard that the journalism industry is enduring a death-of-a-thousand-cuts, which puts us in a situation where an individual's ability to keep tabs on relevant, interesting aspects of government is going to be sustained by the sort of online, data-driven, government transparency web properties that Aaron says are "bunk". In his characteristic style, he gives zero examples, so it's hard to say whether his list of opengov web properties matches mine. But his three possible effects (occlusion, apathy, and misdirection) are all completely orthogonal to the actual issue at hand. People get bored and confused sometimes, government is messy, what else is new? That's why politics is *hard* and why it tends to be the pros and weirdos who show up. It's also why I think your concern is aesthetic - you want politics to stop being difficult, you don't like the fact that keeping up requires pre-industrial effort and attention, and you want to Seymour Hersh and Adrian Holovaty to ride in on a white horse and save us all. Well, Bob Woodward was also a crack investigative journalist and he also ended up being an insider and a shill after it turned out that access was an intoxicating form of currency.

Regarding tech kits for Median Americans, I'm not sure what's out there that can beat the web and a half-assed blog engine like WordPress, seriously for all the features you describe. I mentioned ABetterOakland.com in a comment earlier, and I think it's a bang-on example of what you're talking about and what I'm talking about: came out of approximately nowhere, run by a talented, non-technical communicator with a strong interest in land use policy and an undisguised bias, and fueled by regular attendance at city council meetings and what little web-based government transparency exists here in Oakland. The world doesn't need more tools, it needs more tool users - reporters, page scrapers, republishers, researchers, FOIA sharks, and other openers of black boxes. I liked Robert Caro's Power Broker as much as Aaron, but the fact is that Caro was about 30 years late to the party, and maybe some more timely mechanized/automated disclosure of construction plans in the 1940s and 1950s might have helped the residents of East Tremont keep their neighborhood.

I'm not even sure where to begin responding to your assertion that slow old FOIA can be fixed with a system of ombudsmen and a whole new extra layer of courts between investigators and government - sounds like adding a new shell of armor to the whole thing, further insulating the kinds of people who need to be talking directly with the involved public.

Thomas Lord [27 April 2009 11:18 AM]

Michal,

Well, damned if this isn't indeed an interesting conversation. Maybe that's just me but I think it is anyway.

I'll certainly take some comfort in hearing that I'm saying things that are "completely true but uncontroversial". That's a good starting point. I do understand we differ in how best to draw conclusions from the "true and uncontroversial" stuff. :-)

I think you are unfair in painting Aaron and me as naive rubes who want politics to be "simple". I don't know if you noticed or not (and in some sense I'm successful if you didn't) but this conversation between you and I is a case in point. But, let's move on:

So, in Oakland, you got the PD to start publishing a spreadsheet. There was a real trend, there, in tech efforts for a while: get geographic data like crime reports in machine readable form, make a mash-up using mapping software, and voila: we all a nice visualization / reporting system of that public data. That happened in lots of cities and it continues. Lately, foreclosure and default-notice maps are pretty popular. By tomorrow, if not yesterday, swine flu maps will be big :-)

Honest question: where's the damn impact? I'm sure there is some. I'd bet there was even a measurable if minor impact on real estate prices. Yet as I'm sure you know, Oakland's crime stats haven't exactly turned around. When and where public challenges result in changes to police practices the challenges don't seem to rely much on these new hacks. The murder rate and other violent crime rates suck just as much as ever. The current mess around the Chauncy Baily murder investigation gives some indication of just how healthy and happy the PD is. All the actual progress on these issues is driven by investigative journalism and wetspace grass-roots organizing, no?

Where I see positive change happening around crime and policing issues in Oakland in in Berkeley is where there are old-school organizing efforts and investigative journalism. In my 'hood, we get at least incremental positive changes in policing practices when an outraged community puts enough pressure on to drag the mayor, police chief, and council members down to a community center meeting and confront them. Nobody needs a damn point-and-click map of the police reports because everybody pretty much knows what's going on and, indeed, many know a lot more than any crime map shows.

To make any data from governments relevant and useful (and, yes, of course crime maps can be relevant and useful) -- to do that still takes good old fashioned organizing. Wetspace.

So, I'm not saying your Oakland hack doesn't have impact I'm just saying I don't see it. Show me.

Meanwhile, in terms of tools: You say that the half-assed commercially hosted options aren't worth trying to beat. I'm skeptical about that but willing to play along for the sake of conversation. If we accept your premise there then there is *still* a need to build and promote a "kit" only the "kit" is basically a piece of documentation, aimed at a broad audience, giving guidance about how to use those existing solutions for political organizing. Anyone could do that but the opengov community has a unique opportunity to tie the instructions to government data sources - to link the community building tools to the information sources.

A neighborhood I lived in previous to the current one had a stellar, text-book perfect, "neighborhood disaster prep and crime watch and social communing" group. They do everything from stock-piling blankets and walkie-talkies to getting group-purchase deals on barrels for emergency water supplies to keeping people alert about recent crime patterns to maintaining the garden in a park to holding pot-lucks to holding "take back the streets" kinds of events to.... they are classic, effective, organizers for nothing more controversial than basic peace and prosperity and resiliance. A subset of that group decides they want certain answers from government? They have the clout to push on it and probably get it.

They do indeed use some of the "half-assed" tools of which you speak but they use a lot else besides. People entirely "off-line" get their news from that group from xeroxes at a corner store or fliers dropped off by (informal) subscription. A lot is word-of-mouth. The on-line presence of the group is handy but entirely adjunct to the main value. Software that makes it easy to simultaneously publish photocopied and on-line reports, though - and the smarts to think of doing it that way - is a huge enabler of that group's success. It's such a simple thing, I know, but it isn't *obvious* to most people or even all that easy for people to get the hang of. So, how about a free (libre) documentation guide book and advertising campaign to help spread those tricks and ideas?

As for FOIA and my ombudsmen idea: I'm sure it's a good idea and I have a refinement to it that begins to answer your concerns:

Perhaps the right model here is the public defender system. The courts say that if I'm up on criminal charges I'm entitled to a publicly funded representative in court proceedings. The public defender's office is of government but is formally, professionally, and often even accountably in an adversarial role against various arms of government (especially the police and public prosecutor). It's not perfect but works pretty well.

The "ombudsmen" could be similarly chartered. If my aim is to interrogate my government and there is dispute over what information I am entitled to then I should be entitled to publicly funded, adversarial representation. The rules and regs against which I'm up and knowledge of how to approach the bureaucracies are complex subjects in which its the experts who can be effective. So, let's put some public-benefit experts on the payroll.

And, by the way: thanks. I am, indeed, enjoying and learning from what you've been writing. It's an interesting area and, as you say, politics is hard.

-t

Geeta Dayal [28 April 2009 04:50 PM]

I'm coming from the perspective of someone who teaches journalism at the college level. Aaron and Michal (hi guys!) both make excellent points. But what I think is missing in all of the arguments I've read thus far is a discussion of media literacy and education.

We're watching the print journalism industry literally crumble before our eyes. I'm not being dramatic here. It's kind of scary to watch, like a car crash in slow motion. It's probably even scarier from my vantage point, being a journalist myself. On some particularly dark days, I wonder if teaching journalism is like teaching Greek or Latin -- something from a bygone era, a curiosity preserved in aspic.

In class, we've been building little news sites like Oakland North (oaklandnorth.net) --part of a network of community news sites that have been gaining local traction. We've found that it isn't hard to scoop the San Francisco Chronicle, for instance; we do it all the time.

Right now I work with graduate students with an average age of about 25. I used to teach journalism to 18-year-olds -- college freshmen. When I was teaching teenagers, I noticed that many of them lacked basic media literacy skills. They took whatever they read in Wikipedia at face value. They would look at a graph on a website and not even question the way the data was presented. They weren't skeptical enough about what they read in the newspaper. They didn't look hard enough at the sources of the information. In many cases, their grasp of basic math and statistics was abysmal.

I'm not talking about an investigative narrative journalism vs. online databases dichotomy here. The more that print journalism collapses -- to be supplanted with whatever you want, be it database-driven government transparency websites and data visualizations to citizen journalism initiatives and so on -- the more important it is to teach media literacy. The great thing about a database is you can have access to raw data; my students have more access to more data and more information than ever before. But once you have that access, how do you make sense of it? How do you know what's important and what isn't? How do you know if you're getting enough access? Or if something is being concealed from your view?

This means teaching kids to be critical about what they read, beginning at a very young age, and infusing them with a healthy
sense of skepticism about what they see, both in print, and online. It means encouraging them to do reporting and digging on their own, and showing them how. It means improving how we teach politics and government classes in high school, too. If we can improve basic media literacy among youth, I think we'll be a lot better off.

Michal Migurski [28 April 2009 11:53 PM]

Hi Thomas -

I won't post another essay, but I did want to address the impact of projects like Crimespotting. First, it's not "just" a visualization, it's also a notification system with e-mail and RSS and all that jazz, features that bring information to you if you don't want to visit the site constantly. The nicest thing anyone's ever said about it came from a regular attendee of one of the NCPC (neighborhood crime prevention council) meetings: they used to meet with OPD reps periodically, and hear about the crime statistics in their beat for the first time. One opportunity to see new information, absorb it, and respond. Now, they show up every time with a map and a list of reports so the discussion can immediately progress to a response and a plan. Everyone's better prepared, nobody's wasting their time. The main concerns are stock neighborhood watch business: theft, vandalism, basic quality of life stuff that I *wish* was the only thing the police department had to worry about. The opengov web property is an adjunct and a support mechanism for the existing interpersonal organization, it helps other processes work more smoothly.

Geeta, hello again. =) Do you think the lack of media literacy among 18 year olds is your basic immaturity, or is it something bigger? Maybe I was lucky to get lessons in media skepticism as early as elementary school, but I'm not around enough teens these days to get a good sense for how they thing about media. What you're saying is as basic and important as teaching people to eat right ... media consumption is a form of nutrition, we should know what's good for us. Is there a media parallel to Michael Pollan's "eat food, not too much, mostly plants" ?