Radical Transparency: The New Federal IT Dashboard

Today, at the Personal Democracy Forum in New York, Vivek Kundra, the US national CIO, unveiled the new IT spending dashboards at usaspending.gov. The dashboards are designed to help Vivek and the CIOs of individual government agencies get a handle on the effectiveness of government IT spending.

At the top level, the dashboards provide a view of spending by major government department, with graphs showing performance against schedule, costs, and the CIO’s assessment of how well they are meeting their objectives. For example, here’s a stark view of IT performance at the Veteran’s Administration (click to expand image):

VAToughLove.png

49% of the VA’s IT projects are behind schedule, and Roger Baker, the agency CIO, deems that a full 63% of the projects are in need of serious attention. (Here’s a recent article that outlines Baker’s tough-love plans for IT at the Veteran’s Administration.)

As you drill down, you get to dashboards for individual IT projects (800 projects and approximately $20 billion in budgeted spending). Each project shows the responsible government official, the prime contractors on the project, the CIO’s evaluation of its progress against goals, and each month, an update showing an update of that progress. (We’ll show one of these later in this article.)

The dashboards are an incredibly ambitious undertaking. In the first place, there has never been a government-wide view like this of all IT spending, and the progress of projects. What’s even more remarkable, though, is that the dashboards are being shared with the public. It’s a bit like having your performance review posted on the company bulletin board for all to see.

In notes provided to press in advance of the announcement, Vivek Kundra wrote (italics mine):

Over the past several years, we have witnessed numerous public failures of major information technology systems and just last year saw roughly one third of all investments reported as poorly planned or poorly performing. Many of these investments may never deliver on their original promises. With over $75 Billion in annual federal information technology spending, we need a new foundation for management – one built on the values of transparency, accountability, and responsibility….

Data is powerful. It enables monitoring, reporting, and meaningful analysis that leads to better decisions. Yet, in the case of federal information technology, we lack insight into project performance. Poor data quality coupled with infrequent reporting has led to lack of meaningful analysis and bad decisions. Numerous failures and cost overruns may have been avoided with timely access to accurate information.

The Administration is committed to using technology to move past these barriers. In the IT Dashboard, the public has a platform for unprecedented access to useful, unfiltered data regarding the performance of IT investments. Information available includes responsible government officials and contractors as well as project performance data, updated monthly. This enables better decision-making, giving us the ability to turn around poorly performing projects and to divest from those which no longer make sense.

In making this data publicly available, we are providing unfettered access to investment performance to its true owners – the American people.

Vivek explained that last point further in a telephone conversation with me last night. I asked him about the level of buy-in across the government for this kind of radical transparency about the performance of projects. He said:

“It’s a cultural transformation, in terms of recognizing that we are in the public square. The work that we do is work that is supposed to be performed in the interest of the American taxpayers. And so making visible how we’re performing means fleshing out these complicated issues in the public square. Culturally, making the shift is much better than letting it hide under the veil of secrecy.

Paging through the site, it's clear that not every CIO is on board, or else has not had the time to make the project evaluations. For example, in this project page for the Expeditionary Combat Support System, you can see that the Department of Defense CIO has not yet provided his own evaluation, and even the name of the prime contractor is missing:

FailedProject1.png

According to the Washington Post, some agency CIOs have grumbled about the workload of maintaining the data, but as Vivek noted in that same article:

I talked to the CIO Council and saw the data change overnight. It was cleaned up immediately when people realized it was going to be made public.

That’s part of the new media chess game being played out here.
Having the dashboard public creates an urgency to bring the information it contains up to date, and presumably, to respond to what it shows.
In last night’s phone interview, Vivek noted:

We’re going live in beta, so that we’re giving agencies enough time to make sure that the data and evaluations are up to date. They will have up to 30 days to make that journey.

I love that. 30 days is an eye-blink in traditional government time! We’re starting to see agile methodologies make their government debut.

The site was built on Drupal. Graphics are in Flash, provided by FusionCharts. Based on a reading of the FAQ, it looks like the initial database was generated from data provided by agencies to the Office of Management and Budget (OMB) on congressionally-mandated reporting forms. Monthly updates will be via a web interface provided as part of the project. There are future plans to allow agencies to provide XML feeds.
(It would be good to see that the site uses the same web services architecture to collect its data as it does to provide it out to citizens.)

This is part of being agile, and one of the first lessons in building an “architecture of participation“: rather than having to get complete buy-in from everyone up front, you find a way to bootstrap the project, to get it to what Eric Raymond once called a plausible promise that encourages further participation. What you hope for is the kind of virtuous cascade that gave us successful open source projects like Linux and Apache, and participatory online services like Wikipedia, Craigslist, YouTube, Facebook, Twitter, and, of course, the web itself.

The dashboard platform is primed for citizen participation, not just agency participation. Each page has a “share” icon, which brings up a dialog that lets you tweet a link to that page, share it on Facebook or del.icio.us, or grab the data as an RSS feed or an excel spreadsheet:

Sharing.png

It’s mashup-ready, with a tool for selecting which fields you want in an XML feed. Unlike the main usaspending.gov site, whose API offers 1000 records at a time and discourages repeated calls to download the whole database, the data behind the IT Dashboard is easily downloaded in its entirety. Kudos for that.

Vivek noted:

This is a fundamental change in the way government is going to be run. In the same way that the President was able to mobilize and organize online communities to drive change, what I want to accomplish here is to tap into that energy, to tap into some of the smartest people in the country, accept the fact that we don’t have a monopoly on the best ideas, and make sure that we unleash the online communities of people who are interested in specific areas. Whether it’s environment, or healthcare, or energy, we want them to look at the investments that we’re making. And we want to make sure that we’re getting feedback. A lot of times, the government is not the beneficiary of the best thinking in the country, because something was decided [ten years ago] and we’re not asking, what is the innovative path? How can we achieve our objectives in a much faster way, and at a much lower cost?

In this regard, the view of the management objectives by which each project is evaluated is important. For example, the objectives of this behind-schedule biosense project may already be met by non-governmental projects like INSTEDD or <a href=http://gvfi.orgthe Global Virus Forecasting Initiative – or at the very least, knowledge of this project might allow better cooperation between the governmental and non-governmental projects.

As far as I can tell, feedback right now is limited to a simple comment form. The site would benefit from more structured opportunities for interaction and requests for further information. In particular, we need a better and more detailed view of project objectives for the public to engage properly in oversight and brainstorming. Right now, the project measures what is relatively easy to measure. The next big challenge comes in in saying “what we really need to know in order to be able to decide that the money was misspent is …” and then changing the reporting law or requirements to get that data.

Overall, the data dashboard is a small but important step. Even though the Federal IT portfolio, at $70 billion, is huge, it’s a tiny fraction of overall government spending. But if Vivek Kundra and his team can prove the benefit of radical transparency in this sphere, we can hope that it will spread to other areas of government. We need to see public involvement, so those government agency CIOs who aren’t already on board with the program are encouraged to do so by the flow of attention. We also need to see better overall metrics. What contractors are most often behind schedule and over budget? What departments are struggling? Are we doing better over time as a result of the increased scrutiny?

Dashboards are only as good as the people who use them. Ultimately, the goal of instrumenting government spending and its effectiveness depends on the willingness of those in positions of authority to respond to what the dashboards tell them. Vivek Kundra is rolling out a suite of IT management tools – a platform for better government, if you will – of which this is only one installment. Let’s hope that as the dashboards are updated each month, we’ll see evidence of change, with the data more complete, and with out-of-control projects reined in, improved, or ended.

tags:
  • http://dkmj.wordpress.com Peter Nõu

    Great stuff. We are trying to get the same thing to happen in Sweden, in Europe, but we’re off to a terribly slow start. Sweden is realtively open, but our gov agencies depend on income from selling information (they collectecd through taxpayer financing). How stupid, long term, a funding model is that?

  • http://readwriteweb.com Marshall Kirkpatrick

    Someone asked me today, and I think it’s a good question: is this data also available on data.gov? Would be nice to see data.gov made avail through this kind of interface as well. Would also be nice to see most recent stories from Google or Yahoo News regarding the agency displayed on the page. Perhaps Laconi.ca updates from relevant officials :)

    All that said, three cheers for this advancement towards transparency.

  • Paolo Palombo

    Excellent initiative. It’s always good to use quantitative data instead of subjective judgment; it’s even more remarkable to see it coming from the government.

    I’d like to know more about how data are gathered, and how accurate the information presented is. A dashboard is a fantastic visualization tool, but it is only as effective as the quality of its data sources.

  • http://SiliconANGLE.com John Furrier

    Great post with lots of “data”. Data is the new resource a new “class” of programming has emerged with data at the heart of the value proposition.

    Here radical transparency with open data can yield massive radical transformation. Lets see the wisdom of the crowds to take this data and create new apps to drive society to a better placement one where it is for the betterment of the people not politicians.

    The big threshold issue is will the government listen and endorse new approaches that result from new applications from the wisdom of the people?

  • http://colin-beveridge.com Colin Beveridge

    Tim

    great post. I think that the Federal IT spending dashboard represents a new global yardstick for Government IT spending transparency. An example that should be followed by other countries.

    It’s hardly rocket-science but it would be political dynamite in the United Kingdom, where the government fights tooth and nail against releasing such information – presumably for fear of the outcome.

  • http://www.junitmax.com Kent Beck

    If my math is right, $20 billion / 800 projects = $25 million / project. Do projects that size *ever* go smoothly? OTOH, it would also be a headache to manage 8000 $2.5 million projects. Interesting dilemma.

  • David Locker

    This is cool. I see the challenge of the web site as being able to bring the display and navigation of the site down to a skill level of the average interested citizen. Very cool stuff.

  • http://greg.abstrakt.ch Gregor J. Rothfuss

    I wonder when this level of transparency will come to all parts of the federal government, and then trickle down to states and beyond.

    As bad as IT projects typically go, there are much bigger boondoggles to expose and shut down. Defense contractors, anyone?

  • aaron wall

    -In making this data publicly available, we are providing unfettered access to investment performance to its true owners – the American people.-

    Nice claim in theory. And yet the Treasury and Federal Reserve didn’t want to admit how they were spending our money AT ALL. They sat in congress with bogus “I don’t recall” statements that would make Alberto Gonzales proud.

    Transparency out front while brazen looting is occurring out back is sorta pointless. Its dishonest marketing.

  • http://twitter.com/timekord martin king

    Instrumenting government – this is an encouraging & good first step and once taken difficult to go back.

    No doubt people will “game the system” but at least the game is on now.

    Its all nicely current too – the share button is a smart move indeed and the feeds even more so.

    Berners-Lee wrote some suggestions for exposing Gov data for the UK on the URL below – good suggestion for using RDF but shows how far the UK Gov is behind and still talking…

    http://www.silicon.com/publicsector/0,3800010403,39445943,00.htm?s_cid=111

  • http://tim.oreilly.com/ Tim O'Reilly

    aaron -

    You act as though the government is one entity. It’s 20 million employees. Change takes time.

    This isn’t “dishonest marketing.” It’s some people who are really trying to make a difference, changing the direction of a huge ship. Applaud their efforts and help them, don’t sit on the sidelines and complain.

    In “Mother Night,” Kurt Vonnegut wrote: “You are what you pretend to be, so you’d better watch what you pretend.” That was a warning, but also a guide to better living. Pretend to be better than you are (that’s called aspiration), and maybe you’ll actually start to live up to what you espouse.

    A lot of change starts with idealism about what we can be, not just an expression of what is.

    These guys aren’t accepting the status quo. They are trying to change it.

  • http://www.personaldemocracy.com Micah Sifry

    I was fortunate to be on stage with Vivek Kundra as he demoed the new IT dashboard live at PdF, and frankly I nearly fell out of my chair (and not only because I was leaning backwards to look up at the big screen). Tim is absolutely right; what Vivek is doing is hugely important.

    One point I want to emphasize: when Vivek told the audience that “we’re launching this in beta,” everyone applauded. Earlier, in his talk on “Reinventing Government: What Would Google Do?”, Jeff Jarvis reminded us that if we want government to become more innovative, we have to give government employees permission to fail. Fear of failure (and the “gotcha” moment) too often keeps decision-makers from trying new approaches, but here’s the nation’s first CIO boldly declaring that he knows the site he just launched isn’t fully polished. Kudos.

  • http://info-architecture.blogspot.com Samuel Driessen

    Nice post and I love the initiative. We don’t have this in Holland. You say: “The next big challenge comes in in saying “what we really need to know in order to be able to decide that the money was misspent is …” and then changing the reporting law or requirements to get that data.”
    Very true. What I miss are the big questions these projects are trying to answer. If we have them, we can all help answer these questions. Radical transparency is also about being vulnerable about the stuff you don’t know (yet).

  • http://dialogconsulting.biz Christoph Weber

    The one cool thing I really like: The data includes not only project supervisors, but also contractors. So, we as a people will be able to ferret out consistently underperforming government contractors and call them out on our blogs, report them to our senators and representatives, and in general make clear that we’re watching. This is huge! Feeding at the trough without delivering will be much harder. Just the way it ought to be.

  • http://sites.google.com/site/robotdoctornyc Robot Doctor

    Brilliant first steps…and I am pretty impressed they released this by July.

  • http://twitter.com/govlcy govlcy

    I think this is an excellent effort, however, there has to be continuous momentum to transform and increase transparency within the government through this and other means. In many instances, we prematurely declare “success” then the initiative subsequently arcs and disappears when the “change-maker” departs for greener pastures. Now, if we can do something to hasten the total reinvention of FPDS-NG!

  • Dipanwita Das

    As an Indian, and therefore a member of the largest democracy in the world, it is great to see the US government utilizing technology to introduce transparency and accountability in government spending. As taxpayers, we have always wondered where our money goes.

    Although the introduction and the implementation of the Right to Information Act has brought about a certain amount of accountability in governmental processes, it leaves us hankering for governmental initiatives towards voluntary information sharing with the public.

    In another observation, Tim O’Reilly has mentioned that the charting component used in the setting up of this Dashboard is FusionCharts—an Indian software product. It is ironic that this product, which has contributed towards accountable governance, has so far been un-utilized by the mother country. However, the US government’s steps in this direction are commendable and deserve much recognition.

  • Chris Latendresse

    This is a great tool. It would be great if they exposed the data behind it so intermediaries could mashup the data with other non-gov information, say, private sector ID stats so the gov data can be compared.

    What I found interesting in the IT Dashboard FAQ was the consideration to allow drill-down to the actual Project Manager’s name. As Project Manager, I was trying to determine pro’s and con’s about whether I would want my name posted up with the projects I assigned because many gov projects have a history of tanking or being cancelled or sent into the abyss beyond the PM’s control.

  • http://www.twiki.net Milind Pansare

    What Vivek is campioning is huge. Large companies are just understanding the collaborative enterprise story and moving (somewhat slowly, in part due to the economic woes) to reachitect their IT infrastructure to help track and enable knowledge exchange and transparency. It’s great to see the government CIOs take a lead in this!
    Working for a collaborative platform (TWiki) it’s exciting to be part of this move. Hats off to Vivek!

  • Gary

    I would be very interested in knowing what company OMB contracted with to build the “IT Dashboard” web site. Has this been mentioned in any of the public forums? The IT Dashboard itself is an IT project, and I’d like to know how they managed it and were able to utilize agile technologies in the context of a government contract.