Safari Books Online 6.0: A Cloud Library as an alternate model for ebooks

by Tim O'Reilly | @timoreilly | comments: 120

There has been a lot of attention paid to ebooks lately, and for good reason. Electronic books are portable, searchable, and more affordable than print books. The web has accustomed readers to having the latest information at their fingertips; we all ask why books should be any less available "on demand."

Amazon’s Kindle has received the most mainstream attention (with new entries like Barnes & Noble's Nook making dedicated ebook readers into the latest competitive horse-race), but ebooks are taking off even faster on the iPhone and other smart phones. Ebooks are one of the most popular classes of iPhone application. Recent releases of O'Reilly ebooks as iPhone applications have even outsold the same books in print. Direct sales of the ebook bundles we offer from oreilly.com (PDF, epub, or mobi files) also exceed our direct sales of print books from the site.

Yet our most popular ebook offering by far is often not even thought of as an ebook. Safari Books Online is an online book and video subscription service, launched in partnership with the Pearson Technology Group in 2001. It contains more than 10,000 technical and business books and videos from more than 40 publishers. It has more than 15 million users (including the number of concurrent seats available through libraries and universities); it is now the second largest reseller of O’Reilly books, exceeded only by Amazon.com, and its revenue dwarfs our sales of downloadable ebooks. It's also the most affordable of our ebook offerings for those who are regular consumers of technical content. The average Safari Books Online subscriber uses at least seven books a month, and many use dozens (or even more), yet the monthly price (depending on the subscription plan) ranges from little more than the price of a single downloadable ebook to no greater than that of two or three.

Here’s the rub: most people thinking about ebooks are focused on creating an electronic recreation of print books, complete with downloadable files and devices that look and feel like books. This is a bit like pointing a camera at a stage play and concluding that was the essence of filmmaking!

At O’Reilly, we’ve tried to focus not on the form of the book but on the job that it does for our customers. It teaches, it informs, it entertains. How might electronic publishing help us to advance those aims? How might we create a more effective tool that would help our customers get their job done?

It was by asking ourselves those questions that we realized the advantages of an online library available by subscription. One of the best things about online technical books is the ability to search the full text of a book. How much better would it be to be able to search across thousands of books? Safari Books Online was our answer.

And it just got better. Safari Books Online 6.0, released yesterday, brings a new level of ease of use. It’s a complete, bottom-to-top revamping of the original service. The old UI was, to say the least, getting long in the tooth.

The new UI is slicker and faster, with the kind of drag-and-drop goodness that people expect from a modern web application. In addition, we’ve added some long-requested features, including:

Improved Interactivity -- With 6.0 you can make inline notes, in the actual text you are reading. You can dog-ear or bookmark specific pages. You can highlight text and associate it with notes. When you are done you can print those pages with both your highlights and notes on them. You can scroll non-stop through the pages of a book without any page refresh, or scan a block of pages in thumbnail view to spot the page you are looking for.

Personalized Folders - Rather than having thousands of books and videos organized by us in a single technology topic taxonomy, you can now put together your own organization, grouping books in the categories most useful to you. You can restrict searches to only the books you’ve chosen, and can search within the results of a saved search.

Collaboration - Even better, if you’re a corporate subscriber, you can share your categorization with other members of your company or workgroup. Not only can team members share folders, they can share book reviews, notes and highlights.

Smart Folders - New books, videos and articles are being added to Safari Books Online all the time. Searches saved as "smart folders" make it easy to keep up with the latest content in your area of interest. We have also improved our search user interface to allow you to search inside the book or in other books without leaving the page you are reading. Switch pages only when you find what you want.

As you can see, many of these features take advantage of the online medium in ways that aren’t possible with standalone ebooks. To be sure, there are times you want your own offline copy, and in Safari Books Online, you can indeed download books or chapters for offline use. But especially given the rise of the smartphone as an access device, the times when we are truly "offline" are becoming few and far between. The vision with which we started Safari, that of always-on access to a library of technical content, not just to individual ebooks, is now within reach. Safari Books Online can be used on a desktop or laptop computer or in the browser on a mobile phone. Everything is always in sync because your library is in the cloud.

An ebook cloud works the same way the web itself works. It provides ubiquitous access and shared experience.

Lessons Learned from the development of Safari Books Online

As I outlined above, Safari adopted a "cloud library" model rather than downloadable ebooks as its fundamental design metaphor. I thought it might be worthwhile to understand how we arrived at that decision, as well as some of the other lessons we’ve learned over what is now 22 years of ebook publishing experience. (O’Reilly published its first ebook, Unix in a Nutshell for Hypercard, back in 1987!) With that, a few reflections on lessons learned:

(continue reading)

Thoughts on the Whitehouse.gov switch to Drupal

by Tim O'Reilly | @timoreilly | comments: 43

Yesterday, the new media team at the White House announced via the Associated Press that whitehouse.gov is now running on Drupal, the open source content management system. That Drupal implementation is in turn running on a Red Hat Linux system with Apache, MySQL and the rest of the LAMP stack. Apache Solr is the new White House search engine.

This move is obviously a big win for open source. As John Scott of Open Source for America (a group advocating open source adoption by government, to which I am an advisor) noted in an email to me: "This is great news not only for the use of open source software, but the validation of the open source development model. The White House's adoption of community-based software provides a great example for the rest of the government to follow."

John is right. While open source is already widespread throughout the government, its adoption by the White House will almost certainly give permission for much wider uptake.

Particularly telling are the reasons that the White House made the switch. According to the AP article:

White House officials described the change as similar to rebuilding the foundation of a building without changing the street-level appearance of the facade. It was expected to make the White House site more secure - and the same could be true for other administration sites in the future....

Having the public write code may seem like a security risk, but it's just the opposite, experts inside and outside the government argued. Because programmers collaborate to find errors or opportunities to exploit Web code, the final product is therefore more secure.

More than just security, though, the White House saw the opportunity to increase their flexibility. Drupal has a huge library of user-contributed modules that will provide functionality the White House can use to expand its social media capabilities, with everything from super-scalable live chats to multi-lingual support. In many ways, this is the complement to the Government as Platform mantra I've been chanting in Washington. When you build a vibrant, extensible platform, others add value to the foundation you establish; when you join such a platform, you get the benefit of all those features you didn't have to develop yourself.

Of course, it's easy to imagine that the use of open source software will slash the government's IT budget. After all, this software is freely downloadable. I have a feeling it's quite a bit more complicated than that.

First off, government has a huge number of special requirements (remember the flap over President Obama's blackberry?) Second, don't underestimate the difficulty of doing business in Washington. Procurement is done through a complex ballet understood by few open source companies. Third, a big IT deployment like this requires coordination between many companies, each providing a piece of the puzzle. According to techpresident.com, no fewer than five firms were involved in the switch: prime contractor General Dynamics Information Systems, Drupal specialists Phase 2 and Acquia, hosting provider Terremark, and CDN-supplier Akamai. (Disclosure: O'Reilly AlphaTech Ventures is an investor in Acquia.)

The special nature of the government marketplace is one of the reasons why I launched the Gov 2.0 Expo, which will be held in Washington DC next May. There are huge opportunities for open source, web 2.0, and new media companies in government, but there are also challenges reaching that market. One of my goals for the event is to increase the visibility of cutting edge technology firms not just to government agencies, but also to the prime contractors who are putting together these complex procurements.

The net-net is that I suspect that simply using open source software won't slash government IT budgets, at least not right away. What it will do is increase the amount of value we get for our money and the speed with which new technology can be adopted. Features that would have cost millions of dollars and years of development to add will now be rolled into the scope of current contracts.

It's also important to realize that using open source is very different from contributing to open source. Despite the exaggerated claims in the AP story, that "the programming language is written in public view, available for public use and able for people to edit", the White House has not yet released any of the modifications they made to Drupal or its operating environment back to the open source community. The source code for Drupal (and the rest of the LAMP stack) is indeed available, but the modifications that were made to meet government security, scalability, and hosting requirements have not yet been shared. In my conversations with the new media team at the White House, it is clear that they are exploring this option.

Giving modifications back to the Drupal community is the next breakthrough announcement that I'll be looking for.

Releasing code is more than just being a good open source community citizen, though. Code sharing is a major cost-saving opportunity for government. There are countless government agencies at the federal level, not to mention at the state and local level, that perform similar functions. Yet each of them does its own development, driving up costs. Federal CIO Vivek Kundra has made a great step forward in web services by creating data.gov. I'm eager to see an analogous code.gov portal for government agencies to share their open source software code.

Web 2.0 Summit Starts Today

by Tim O'Reilly | @timoreilly | comments: 1

Last year at Web 2.0 Summit, one prominent tech executive responded to our focus on "Web meets World" -- the way web technology is being used to attack the world's problems -- by saying "I don't come to this conference to learn how to do good. I come to learn about trends that are going to affect my business."

As it turns out, the "Web meets World" theme was in fact exactly on point with the trends that were going to affect his business. What Fred Wilson calls "the golden triangle" of Web meets World trends -- mobile, social, and real-time -- are at the heart of many of the cutting edge non-profit activities we showed last year, and they are very much at the heart of the for-profit companies following hard on their heels.

I've written a much longer paper on this subject - Web Squared: Web 2.0 Five Years On, and I won't repeat that there. But that's the theory. The practice is how entrepreneurs are taking advantage of these disruptive trends, how big companies are responding, and what kind of infrastructure changes we'll need to support the future that is coming at us.

This year at the Web 2.0 Summit, we'll be hearing how real-time, social, and mobile play out in the strategy of Google, Microsoft, Intel, Facebook, Twitter, Yahoo!, News Corp, AOL, Comcast, Nokia, and even GE, but we'll also be hearing from entrepreneurs, and yes, even some more innovative hackers who are helping birth the future away from the commercial limelight.

The official sessions are great, but it's the hallway conversations that can really set your mind off in a new direction. For example, at a pre-Summit event last night, I had a fascinating conversation with Marc Pincus of Zynga last night about his belief that the third great internet business model has arrived. Fortunately, you don't need to bump into Marc to hear what he thinks: he's speaking this afternoon at 4:15. He's put his ideas about social selling into practice, with 129 million users playing Zynga games each month, spending millions of dollars on virtual goods. But what's most fascinating is how Marc sees the potential to apply social gaming principles to all of e-commerce. His riff on how what's he's learned applies to Amazon (and anyone else selling on the web) is worth the price of admission to the Summit.

I hope to see you at the Summit. John Battelle and I kick off the show with opening remarks at 2 pm at the Westin Market Street in San Francisco.

Blog Action Day 2009: Climate Change

by Tim O'Reilly | @timoreilly | comments: 13

Today is Blog Action Day 2009. This is an annual event, held every October 15, with a goal of encouraging an outpouring of simultaneous comment on an important issue calling for global action. This year, the designated subject is climate change.

Back in January, I wrote a blog post summarizing my position on climate change. Entitled Pascal's Wager and Climate Change, the post makes the argument that even if you're a skeptic about climate change or humanity's role in causing it, the risks of ignoring the issue are great, and the benefits from addressing it are significant even if scientists are completely wrong about the causes. What I wrote in January still seems sound, so please go back to the original post, linked above, to review my argument and the vibrant comment thread.

In the meantime, here are a couple of my favorite climate-change related resources:

• Greenmonk. Greenmonk is a good blog, but I also love their mission of providing advisory services to companies trying to develop climate change strategies.

• RealClimate, which bills itself as providing "Climate science from climate scientists", and delivers.

• energyliteracy.com, a site created by my son-in-law Saul Griffith to help people understand the math and engineering concepts around energy use and climate change. It's amazing how many people talk about the issue without understanding the basic units with which energy is measured. Wattzon is another site that Saul created to help people quantify the energy they use. From what I can see, users way under-report their actual energy consumption, but the ideas, presentations, and posts on the site are extraordinarily informative.

• Worldchanging, a site that doesn't just cover climate change, but focuses on technologies and practices with positive global impact.

Feel free to supply additional links (pro- and con-) in the comments. And if you care about the issue (you should!), write your own Blog Action Day post.

My Conversation with Austan Goolsbee at Web 2.0 Summit

by Tim O'Reilly | @timoreilly | comments: 8

He introduces himself as "another tall, skinny guy with big ears and a funny name." Economics adviser to Barack Obama during the campaign, and now a member of the President's Council of Economic Advisers and chief economist of the Economic Recovery Advisory Board, Austan Goolsbee is a key figure in framing the economic thinking of the Obama administration. Perhaps most importantly for those of us in Silicon Valley, he's an economist clued in to the tech world. His economics papers cover such topics as the impact of taxes on technology diffusion, the impact of internet subsidies on public schools, and the economic impact of leisure time spent on the internet. He's worked closely with Richard Thaler and Cass Sunstein of Nudge fame, and thinks a lot about the power of default options to shape behavior, a topic that any web developer should also know by heart.

I'll be interviewing Austan Goolsbee on stage at the Web 2.0 Summit. In preparation for our conversation next week, I spent an hour with him yesterday morning. He's a fascinating guy. To give you a taste of the kinds of things we'll be talking about, here's a short transcript of his response to my question about the economic impact of the internet:

Somehow, in my economist heart always lies the revealed preference thing, which is: People are investing tons of their time, tons of their money, tons of their energy into the internet; they wouldn't be doing this for no reason. Regardless of whether we have the data, our presumption ought to be that it's a big productivity improver. But I also think that the evidence on big general-purpose technologies like that is usually that when they're first invented, the impact takes a while to show up, but when it does, boy is it a big time thing, outside just the industry itself, across the board.

If you go back ten years, which isn't that long, the social landscape and the technological landscape are almost unrecognizable. And just that impact, at this early stage, is sufficiently big that you've got to think that twenty years from now, the internet is going to have humongous productivity implications.

Take the health sector. People say "not only does the health sector need to enter the 21st century, it needs to enter the 20th century!" The technology is sufficiently backwards in terms of the information processing - everything's on paper! If you start envisioning healthcare, energy, the government itself -- major league shares of the GDP -- and what the potential is of marrying that to the newest technologies...! Economic potential, historically speaking, tends to be a bit like water. Water will always get to the lowest point. If there's big potential somewhere, it may take a bit of time, but we always find a way to unlock it.

We'll be talking about what Goolsbee would recommend doing differently if we had a "do-over" on the economic stimulus, the importance of innovation to any future economic recovery, education and income inequality, financial services oversight, and President Obama's desire for "iPod government" (which Goolsbee describes as "making [government] simple and easy to use, so that people like it, rather than giving people the third degree and a lot of red tape.")

If you had a chance to sit one-on-one with one of President Obama's economic advisers, what would you ask him? Help me prep for the interview by making suggestions in the comments. It will be tough to do as good a job as Jon Stewart, but hopefully we can come up with some questions that get Austan going!

"He not busy being born is busy dying"

by Tim O'Reilly | @timoreilly | comments: 38

I found myself quoting that great Bob Dylan line the other day on a mailing list for those dealing with the changes sweeping through the publishing industry. Michael Coffey from Publisher's Weekly wrote an eloquent and moving lament that expresses the fear of many that the book might be losing its pre-eminent position in the cultural canon. He wrote:

I think one of the perspectives little addressed in this terrific thread is that of the impact disappearing books might have on the writer---not the librarian, bookseller, or reader--but the creative class for whom leaving behind, in their stead, on bookshelves or libraries of in the collections of their families, a discrete object that is their creation, [is] a testament to the occcasion of their having created something out of nothing. This creation of an artifact might, I would propose, have played an enormous part in our cultural production over the centuries. To know that there, in a little dimensional space somewhere are one's singular compositions-- bound and protected (to some extent) and real--has power. If the book disappears to a degree that allows us to say, "hey, hasn't the book disappeared?--how will culturual production change? Will knowing your work is in code in some nondimensional space, dark and immaterial unless accessed by a curious soul, be enough to replace that other enticement?

Much as I sympathize with Michael's concern - I love books, make my living publishing them, and live surrounded by them as my constant companions - and despite loving the way he framed the question, I had to take issue with the idea that if books became less important as artifacts and carriers of culture, writers would stop writing:

Given that cultural production is continuing unabated on blogs and YouTube, I don't think we need to worry about it "disappearing" but you're right that it will likely change. But who says that the book has ever been the ideal unit of self-expression, or the best tool for expression of ideas?

Michael replied:

I do think centuries of cultures have held that belief--perhaps not that it was the ideal unit, but the best we had. Now, of course, the issue of ease of access has become paramount, and the traditional book has slipped a bit further back from the "ideal." But it reigned supreme for a long time for good reasons, and its status as an object was part of it. I guess it is our privilege to see how this may work itself out...

Michael makes a really good point, that as we move from books to ebooks, and to other forms of writing - on the web, for mobile devices, and even on evanescent media like mailing lists and twitter - the way we express ourselves will also change. But of course, this is not new. The "modern novel" is only a few hundred years old; poetry was once primarily a spoken art and written down only after the fact. You have only to read an eighteenth century book to realize how much the form has evolved even during the tenure of the printed and bound book. I responded:

Well, for many centuries, the painting played that role in visual arts. It was superseded (for many purposes) by photography. Painting still is practiced, but it's a niche art. However, visual arts are practiced more widely than ever before, and are appreciated more widely, by more people. Something is lost, but something is gained.

Worth reflecting on: the opening chapters of Victor Hugo's Notre Dame de Paris ("The Hunchback of Notre Dame") are a lament to the loss of visual literacy - the ability to read the stories told in the sculpture on the walls of a Cathedral - as a result of the rise of reading.

We might also lament the loss of the prodigious feats of memory that were common before literacy was widespread, or the loss of physical skills (horseback riding, shooting, fencing, needlepoint, piano playing) that were once common before the rise of the technologies that made them hobbies rather than necessities.

A great line from Kim Stanley Robinson's Mars trilogy: "He knew then that history is a wave that moves through time slightly faster than we do." Eventually, we all get out of step with the world as the habits of our youth are replaced by the habits of the next generation's youth.

P.S. Another great novel dealing with this effect is Booth Tarkington's The Magnificent Ambersons. Just as the motorcar ended the days of the horse (the frame of that story), new forms of writing and reading may be ending the days of the modern book. This is a good thing, overall, if we embrace it. We can be vaudeville players sneering at the new moving pictures, or we can go west and become part of the new Hollywood.

Transitions are tough on old industries. But remember that part of why they are tough is that something new is being born. Bob Dylan: "he not busy being born is busy dying."

Going back to check my facts, I find that it is not an early chapter of Notre Dame de Paris, but later in the book, in a chapter entitled One Shall Destroy The Other, that Victor Hugo explores the relationship of the cathedral and the book:

This thought is full of the foreboding that one power is about to be succeeded by another. "One shall destroy the other;" in other words, the Press shall overthrow the Church.

But underlying this thought — the first and simplest, it is true — there is...the presentiment not merely of the priest, but that of the scholar, the artist; the presentiment that human thought, in changing its form, was about to change its mode of expression ; that the leading idea of each generation would not always be inscribed in the same fashion, with the same material; that the book of stone, so solid and durable, was about to give place to another, still more substantial and durable, — the book of paper. Beneath the archdeacon's statement, his vague formula, there was another, deeper significance; the thought that one art shall dethrone another, — Printing shall 'overthrow Architecture.

It's a brilliant chapter, full of humanity and wisdom. Anyone who is afraid of change, and needs context to accept it, should read it.

As authors and publishers explore the new world of online reading and writing, we need to do more than just translate print books to an electronic screen. We have a future to invent! And the time is now. At the just-concluded Tools of Change for Publishing Conference in Frankfurt, Sara Lloyd of Pan Macmillan drove home the urgency with which publishers need to engage with the grand experiment:

"The new thing is never as good as the old thing, at least right now. Soon, the new thing will be better than the old will be. But if you wait until then it’s going to be too late."

I have a feeling that this is a lesson not just for publishers, but for our entire culture, as many of the things we've taken for granted begin to shift, either because of new inventions, or because of failures of existing systems.

"He not busy being born is busy dying."

Microsoft Press Enters Strategic Alliance with O'Reilly

by Tim O'Reilly | @timoreilly | comments: 31

Today, Microsoft and O'Reilly Media announced an agreement to support and expand Microsoft Press. Under the terms of the strategic alliance, O'Reilly will be the exclusive distributor of Microsoft Press titles and co-publisher of all Microsoft Press titles, on Nov. 30, 2009. We'll be working with Microsoft to develop new books, as well as distributing both existing and new co-published books to bookstores, and, perhaps most importantly, to the emerging digital book channels that represent the future of book publishing. Microsoft could have chosen to partner with any of the major computer book publishers. That they chose to work with us is a testament to three advantages we bring to the business:

1. O'Reilly is more than a book publisher. We are an advocate, a connector, and a community builder. We help developers and users make the most of technology, with a focus on what they need to know. Microsoft has a history of building great developer communities, but in today's world, those communities need to be connected with other communities outside Microsoft. Especially in technology, "the world is flat."
2. O'Reilly plays a unique role in the technology ecosystem: from our earliest days, we provided the documentation for important technologies for which there was no "vendor." The internet, the World Wide Web, Linux and other open source software, and Web 2.0 all were documented and given mainstream awareness by O'Reilly books and events. We identify and evangelize the disruptive technologies that reinvigorate the industry.
3. O'Reilly has been a pioneer in the new world of ebooks. In the early 1990s, we co-developed docbook, one of the first standardized formats for ebooks, and the progenitor of future XML-based ebook formats. In 2001, in partnership with the Pearson Technology Group, we launched Safari Books Online, the largest and most comprehensive electronic subscription library of computer books and videos. We've built a successful direct business with DRM-free downloads of ebook bundles that work on any device. We're an early leader in publishing books for the iPhone and other portable reading devices, and understanding how to use ebook channels to reach new customers. And of course, our Tools of Change for Publishing Conference (TOC) has become the place to share knowledge about the changes sweeping through publishing.

On this last point, I'm particularly excited that as part of this agreement, Microsoft has committed to make its ebooks DRM-free and device-independent. One of our goals at O'Reilly has been to make sure that ebook customers can read them on any device, and have the ability to keep using them even if they change their preferred device. Having Microsoft Press join us in this commitment is a big step forward towards an open ebook market.

What Does Government 2.0 Mean To You?

by Tim O'Reilly | @timoreilly | comments: 20

As many of you know, I've built a new conference, Gov 2.0 Summit, around the idea of the government as platform: how can government design programs to be generative, to use Zittrain's phrase? How do we get beyond the idea that participation means "public input" (shaking the vending machine to get more or better services out of it), and over to the idea that it means government building frameworks that enable people to build new services of their own?

I've been talking a lot about this topic recently, so there are plenty of places to see and hear what I think. (Here are links to my Forbes column on Gov 2.0, an interview with Marshall Kirkpatrick of ReadWriteWeb, my interview with Mark Amtower on Federal News Radio, and my Radar talk at this year's OSCON. (Gov 2.0 remarks start about 9:45 in, with my idea of what Gov 2.0 is really about starting at around 16 minutes.)) And in a few weeks, you can hear the latest thinking from some key people in the world of policy and technology at Gov 2.0 Summit.

But I'd like to reach beyond the voices of the people on stage at that event, and include your voices. So I'm throwing out an invitation in the form of a question: what does Gov 2.0 mean to you? The question is intentionally open to interpretation in a variety of ways, so go to town!

I'd like to hear from you through short video clips; just tag them with #whatisgov2 and post them to your favorite video service by September 2. Details about the video invitation are here. I'll take some of the best of these videos to Gov 2.0 Summit in two weeks, and make them part of the conversation there.

Seeing Our Culture with Fresh Eyes

by Tim O'Reilly | @timoreilly | comments: 40

The other day, I read a novel called Prester John, by John Buchan, published in 1910. This story about a Zulu uprising in South Africa as experienced by a young Scottish immigrant is an entertaining read in the spirit of Rudyard Kipling or H. Rider Haggard: adventure in the furthest outposts of the British Empire.

But what makes this book most worth reading today is how many things the author takes for granted that we now know aren't so, and even find distasteful. The racism of the book is shocking precisely because it is so casual and thoughtless, the innate assumption of white superiority.

It makes me wonder what people a hundred years from now will think of our popular fiction, our popular movies. What do we take for granted that they will find odd, and perhaps even distasteful? You can already see some obvious candidates in things that are still accepted, but barely, like smoking. How curious it is to see a movie in which everyone is puffing on a cigarette - for example, in Good Night and Good Luck, where Edward R. Murrow is shown delivering prime time television news with a cigarette between his fingers.

What will people think of our enormous steak dinners and obese portions of food? That's on the cusp of changing. What will they think of our profligate use of fossil fuels and other non-renewable resources? Our assumption that the American way of life will go on forever, just as it is, much as the British thought their empire would go on forever? What about our assumptions about unlimited technological progress? Will science fiction visions of star flight or "the Singularity" seem as quaint as "the White Man's Burden"?

Above all, what will they think of the appalling amount of waste in our culture? Have you ever walked through a tourist area - say Fisherman's Wharf in San Francisco - and seen entire stores devoted to schlock, made in developing countries by people who must scratch their heads in wonder at a people so wealthy that they can afford to spend money on things that are so utterly and obviously useless?

But even the stuff that is useful is the product of a world that is as outdated and unnatural as the colonialism of the British Empire. We live in a throwaway culture sustained by sweatshop labor, which has replaced a culture of heirloom products that last generations. And that's progress? (See Saul Griffith's thoughts on why owning products that last a lifetime is an important part of going green.)

In this regard, I urge everyone to read Fake Steve Jobs' amazing column about the Foxconn employee who committed suicide after losing an iPhone prototype, I'm really thinking maybe I shouldn't have yelled at that Chinese guy so much. Nat Torkington just quoted this piece in his Four Short Links for today, but the quote he chose is so appropriate to the post I have been writing that I just have to include it:

We all know that there's no fucking way in the world we should have microwave ovens and refrigerators and TV sets and everything else at the prices we're paying for them. There's no way we get all this stuff and everything is done fair and square and everyone gets treated right. No way. And don't be confused -- what we're talking about here is our way of life. Our standard of living. You want to "fix things in China," well, it's gonna cost you. Because everything you own, it's all done on the backs of millions of poor people whose lives are so awful you can't even begin to imagine them, people who will do anything to get a life that is a tiny bit better than the shitty one they were born into, people who get exploited and treated like shit and, in the worst of all cases, pay with their lives.

Not a pretty picture. But sometimes a look in the mirror is a good way to wake up and change your life.

We're in the middle of a global economic downturn. Many of us imagine that our goal is to get things back to the way they were. I believe it's an opportunity to imagine a better future, to build an economy that is more robust and more fair than the Ponzi economy of the last fifty years.

Radical Transparency: The New Federal IT Dashboard

by Tim O'Reilly | @timoreilly | comments: 21

Today, at the Personal Democracy Forum in New York, Vivek Kundra, the US national CIO, unveiled the new IT spending dashboards at usaspending.gov. The dashboards are designed to help Vivek and the CIOs of individual government agencies get a handle on the effectiveness of government IT spending.

At the top level, the dashboards provide a view of spending by major government department, with graphs showing performance against schedule, costs, and the CIO's assessment of how well they are meeting their objectives. For example, here's a stark view of IT performance at the Veteran's Administration (click to expand image):

49% of the VA's IT projects are behind schedule, and Roger Baker, the agency CIO, deems that a full 63% of the projects are in need of serious attention. (Here's a recent article that outlines Baker's tough-love plans for IT at the Veteran's Administration.)

As you drill down, you get to dashboards for individual IT projects (800 projects and approximately $20 billion in budgeted spending). Each project shows the responsible government official, the prime contractors on the project, the CIO's evaluation of its progress against goals, and each month, an update showing an update of that progress. (We'll show one of these later in this article.)

The dashboards are an incredibly ambitious undertaking. In the first place, there has never been a government-wide view like this of all IT spending, and the progress of projects. What's even more remarkable, though, is that the dashboards are being shared with the public. It's a bit like having your performance review posted on the company bulletin board for all to see.

In notes provided to press in advance of the announcement, Vivek Kundra wrote (italics mine):

Over the past several years, we have witnessed numerous public failures of major information technology systems and just last year saw roughly one third of all investments reported as poorly planned or poorly performing. Many of these investments may never deliver on their original promises. With over $75 Billion in annual federal information technology spending, we need a new foundation for management - one built on the values of transparency, accountability, and responsibility....

Data is powerful. It enables monitoring, reporting, and meaningful analysis that leads to better decisions. Yet, in the case of federal information technology, we lack insight into project performance. Poor data quality coupled with infrequent reporting has led to lack of meaningful analysis and bad decisions. Numerous failures and cost overruns may have been avoided with timely access to accurate information.

The Administration is committed to using technology to move past these barriers. In the IT Dashboard, the public has a platform for unprecedented access to useful, unfiltered data regarding the performance of IT investments. Information available includes responsible government officials and contractors as well as project performance data, updated monthly. This enables better decision-making, giving us the ability to turn around poorly performing projects and to divest from those which no longer make sense.

In making this data publicly available, we are providing unfettered access to investment performance to its true owners - the American people.

Vivek explained that last point further in a telephone conversation with me last night. I asked him about the level of buy-in across the government for this kind of radical transparency about the performance of projects. He said:

"It's a cultural transformation, in terms of recognizing that we are in the public square. The work that we do is work that is supposed to be performed in the interest of the American taxpayers. And so making visible how we're performing means fleshing out these complicated issues in the public square. Culturally, making the shift is much better than letting it hide under the veil of secrecy.

(continue reading)

Recent Posts

• My 140conf Talk: Twitter as Publishing on June 24, 2009
• A Manifesto on Health Data Rights on June 22, 2009
• Velocity: The Art of Web Operations on June 22, 2009
• The Benefits of a Classical Education on June 21, 2009
• Health Care Costs: Am I missing something? Or is there a lot of flimflam going on? on June 19, 2009
• Personal Democracy Forum: Politics in the Web 2.0 Era on June 19, 2009
• Jeff Bezos at Wired Disruptive by Design conference on June 15, 2009
• Google I/O in Pictures: Google Culture at Work on May 29, 2009
• Google Wave: the Early Days on May 28, 2009
• Google Wave: What Might Email Look Like If It Were Invented Today? on May 28, 2009