My mind is buzzing right now…

My mind is buzzing right now, full of ideas that are all demanding my attention, begging to be written up. I don’t have time right now, but I thought I’d write down the headlines, both as a kind of “to do list” and also to give a sense of the stew of topics I’m spending time on:

  • Why needs to use revision control
  • Pathologies in the hive mind
  • Why we need cloud data services, not just cloud database services
  • Why our energy discussions will falter till we’re all using the same math

  • My priority list for technology initiatives in the next administration.

Hopefully I’ll get a chance to write up all these posts this weekend, and get them out in the coming week.

tags: , ,
  • Heh, yes, was thinking _exactly_ the same thing about

    There was an article recently (NYT, I think) about how some material had disappeared from the site, only to reappear again a bit later retooled — basically, they took the campaign rhetoric and softened it down for Which is fine, but having those changes be auditable, like Wikipedia’s, would be so, so much better.

    I do think they get the tune, but sometimes they haven’t quite gotten all the words yet. For example, during the campaign, the Obama-Biden technology platform had a great section on better, more parseable access to government information. But the site itself didn’t have a search box!

  • Cee Bee

    Good grief! Save the nitpicking and Google the site by appending to your search string.

  • Hi!

    We need distributed revision control :)

    We do not want information about “who edited what” to be lost in a single repository. Give us line by line changes where we can see how groups built the final proposals before they were merged into an overall bill.

    Distributed revision control for bills gives more ownership to groups and provides for more accountability.


  • Jeff

    “Why our energy discussions will falter till we’re all using the same math”
    Agreed since most of the doom and gloom crowd ignores science.

    “My priority list for technology initiatives in the next administration.”
    Here’s mine: HANDS OFF – GO AWAY AND LEAVE US ALONE. The chance of it happening? Zip and zilch.

  • In regards to “Why our energy discussions will falter till we’re all using the same math”, I just saw a movie called fuel the other day which was an examination of our dependence on petroleum and how it got that way. It really showed the hidden costs of using fossil fuels, things like the costs of cleaning spills, ultra-high rates of cancer in areas close to refineries, the subsidization of the price of oil through our military operations, relief for global warming related disasters, etc. To summarize the movie, there really is a lot of potential in BioDiesel, which work fine with existing diesel engine technology. It’s already taking a strong hold in europe. BioDiesel engines run cleaner and are renewable, and the price really can be competitive. Ethanol isn’t the right solution, but was unfortunately forced through via the force of the farming lobby. The real breakthrough that we should be hoping for is in algee.

  • All right… wasn’t going to ask, but “pathologies in the hive mind”? What what?


  • Ken Roberts

    Nothing like a buzzing mind.
    Thanks for sharing

  • Data services in the cloud are coming together pretty quickly, at least in the geospatial world. There’s an interesting mix of large and small companies working on the problem. I think well see some seriously interesting capabilities in 2009.

    The big constraint right now centers around finding a consistent way to let users integrate their own data with cloud data.

  • Tim,

    As a test for your “Energy Math” project – take a swing at figuring out the “total” energy required to build some of these “energy saving devices”. My math say many things we think of as “energy saving devices” are actually burning more total energy – it’s just burned building it rather than running it.


  • Great ideas. Got me thinking.

    My main issue with is that its code is not open. They need to use something like trac and release it as public domain open source project. Of course, you get great version control with the package.

    The hive mind is broken because we’re not bees. Our communities peek at 150 members. We have to let go of the dream of global control of the markets and focus on our local market.

    Why we need cloud data services, not just cloud database services? because data services are more general, supporting more then just the old SQL.

    Thanks for sharing.

  • Brian (who wrote “We need distributed revision control. :-)”),


    Well, yes, we do. But there is a problem that will keep the idea from taking meaningful hold someplace like or with someone like Karl: they are invested (variously socially and economically) on a bet that we don’t need distributed revision control. Conversely, if they can sell an off-their-shelves modification to, or float an internally used one to prominence, the market value of their social and economic investments go up. Queue the “good enough for government work” arguments…

    It’s also hard to sell because those who know the term “distributed revision control” often disagree about what it means and, anyway, most people don’t know the term. I like to express it in more familiar terms:

    If there is a draft of some government document (or, really, any kind of document) floating around the net, with people suggesting changes or alternative versions in this or that form — well, that’s an awful lot of subtly but importantly different versions floating around. Where sunshine laws apply, some (not necessarily all) versions that float around — say in emails between officials — are “of record”. By “of record” I mean that they need to be archived for public access.

    Public access requires, at the very least, that the archive of all of those versions ought to be “indexed for easy reference”. By “indexed for easy reference” I mean that every variation of the document — every “changed version” that someone sent around in an official capacity in a sunlight situation — needs an official name.

    The official name of some variation of a document ought to be at the very least “human-friendly enough” that it can be used, for example, in a foot note. And, given the use of an official document-version name in a foot note, it ought to be possible to ask a good library to accurately retrieve the exact document referred to (given just the official name).

    Note that we’re not just talking here about one “official archive”. Every library — every person, indeed, — should be able to be an archive, if they so choose. Whatever the “official name” of each document is, everyone has to be able to use that name as an index into their copy of the archives.

    And “official names” have to be human friendly in the sense that a person can somewhat casually memorize them and recognize them by sight. Otherwise, corruption of records will no doubt abound.

    And a subtle point about official names: we want sunshine but we also want freedom of speech. Any official ought to be able to create and publish a new version of some document, under sunlight conditions, without having to petition some central authority for an Official Name for the document. The creation of new official names has to be distributed.

    Let’s assume we have official names for documents and variations on those documents. What are the relations among these?

    For example, a typical “variation on proposed legislation” would be generated when a legislator gets one version of the legislation, has staff make some changes, and sends out the changed version. The original and the changed version have official names but the record also reflects that the second version is a direct “reply to” the first version and the record, carefully examined, let’s us see exactly what changes the legislator’s staff made. This is significant in political discourse:

    How many times have you read in the paper, in the blogs, or seen on TV a claim like “At the last hour, congressman X snuck in a provision that…”?

    Snuck in? Well, in any event we now have a very common question which is how two versions of documents differ and another very common question of when, exactly, each modification was made to a given version of the document.

    We can give official, human-friendly, “distributed” names to those changes, as well.

    What’s in a name? With a system of naming we can begin to construct an open-ended collection of document management software that uses those names and interoperates accordingly. So, there could be lots of different, sometimes even competing tools for examining the public record — all referring to an objective, neutrally but usefully indexed record.

    Once we give distributed names to versions of documents, relations between documents, and the changes between docments — and start to build interoperability standards for programs that use those names — we have what the geeks call “distributed revision control”.

    To go back to geek mode:

    One application that simply “falls out” of distributed revision control — a “free bonus” — is a global, distributed, highly robust, transactional file system based on write-once / forget-and-remember-at-will tertiary storage. The requirements of such a “miracle” file system are the same. The difference between distributed revision control as understood by programmers using it for source code and a global, transactional file system is nothing more than choices about what kinds of usage patterns to optimize for — and you can have both at once, seamlessly.

    Some day, perhaps sooner than I would have hoped after the miserable experience of the GNU Arch project, we’ll mainly have applications where the menu item “Save” is understood to mean “add a journal entry” to the global, transactional, distributed-journaling file system whose system of file names would, for example, make it trivially easy to examine the history-of-record of, say, the final text of a piece of legislation.


  • Hi Tim

    Can you elaborate on “Why we need cloud data services, not just cloud database services”?


  • river

    Alright Juha. If we were friends on Facebook, I would stand up first in the ovation and send you a Blue Vodka Meme. Then I would disappear. I’m either going to share that link, or keep it to myself and savor it. It is deeply relevant either way.

  • (I probably shouldn’t, but…)

    Tom, your conjecture about my investedness in a particular form of revision control is wrong.

    I’m not sure why you think I’m opposed to distributed revision control for or similar situations. My comment certainly says nothing to that effect. The rest of your comment makes sense to me, and it’s how I see the world too.

  • Karl,

    You wrote:

    (I probably shouldn’t, but..)

    Oh, but you should. I was not trying to pick a fight with you but to bait you. What I mean by that is:

    If the topic here on radar results in no economic activity then I’m not proved right or wrong but my hypothesis stands. On the other hand, if the economic result is anything from the folks exposing their existing centralized revctl archives for view to adopting a centralized system of revctl and exposing that then my hypothesis receives a nice conformation. And on the third hand, if we actually generate investment in this project we’ve talked about off and on for, what is it 6 or 7 years now?… if that happens then my hypothesis is refuted.

    I would think, especially if it the investment involved me, that I would have nothing to complain about having my hypothesis refuted. Please, by all means, prove me wrong!

    I’m not sure why you think I’m opposed to distributed revision control for or similar situations.

    I’m not sure I do think that you are “opposed”. I just doubt that, if economic activity is generated as a result of this chat here, that that activity will try to address the distributed revision control needs and potentials we seem to agree upon. Far more likely, I think, is that if anything were to happen at all it would be the application of an off-the-shelf solution. For example, perhaps might hire Collabnet. Everyone will slap each other on the back and then what I call the “good enough for government work” arguments will appear in the sense of the usual cheer-leading about the wisdom of crowds, and the shallowness of bugs given “many eyeballs” etc. That we can point out here a more rational, more profound kind of investment to make in developing new technology that will be like “dust in the wind” in terms of its real world impact — just as have been all similar conversations with the big dogs in this niche industry since about January of 2002.

    I know of no simpler explanation for that history of failure to act than the economic one: at each juncture people are protecting their sunk costs in contrary technological approaches. People acknowledge the better potential of the other approach, perhaps mumble kind words about how they look forward to getting around to working on it after the next big milestone — and then the topic is forgotten and neglected.

    Try and point out this failure mode in our “community” and what do you get? Feedback like “(I probably shouldn’t, but…)”. The more things change, eh?

    The rest of your comment makes sense to me, and it’s how I see the world too.

    How do we muster a sustainable and sustained effort, then?

    It’s apparent to me (though I don’t claim you should be convinced by that alone) that this system we’re talking about isn’t going to evolve out of bazaar, mercurial, git, arch, or any of those projects. Each of those brings some excellent insights into the problem to the table but each also has a hefty list of flaws and heavy sunk costs in those flaws with no practical path for fixing them.

    At the same time, trying to do it by gathering luminaries from those various projects around a table and asking for consensus doesn’t strike me as likely to produce anything worth producing anytime soon — that’d be a symbolic, ineffectual way to separate out the right ideas and combine them.

    It’s easy enough (wouldn’t you agree, can we stipulate?) to argue that successful and effective investment in this system would grow the market for free software support services (and similar) by far more than the size of the investment and yet it is not superficially obvious how any particular investor could expect to see a winning share of that return.

    It could make for an interesting case study in “free software R&D” to take that investment problem as the technical problem to be solved. Not a technical programming problem but a technical trading problem for the free software industry. I have a pretty good idea how to fix that problem but my first assumption is that nobody will listen and my second assumption is that somebody listens they’ll repeat history and “rip me off”, so why should I bother…

    So you see where I’m coming from. By all means, please, prove me wrong.


  • Sam Penrose

    Carbon / GHG accounting seems to be as difficult as it is important. I’m fooling around with a visualization-based approach in my spare time. If anyone reading this is working on it and can use a few cycles from a programmer, my email domain is and my username is sampenrose.

  • bowerbird

    thomas lord, you should build the system you describe.

    the world has needed something like this, ever since
    ted nelson first laid out the idea a very long time ago.


  • Looking forward to your post on “cloud data services” versus “cloud database services”.

    If I could venture a guess about the distinction: databases are software for serving data, but it’s the data that’s the good stuff.

    It’s great that Vertica is launching an EC2 offering of its database. But what the world needs is not databases living on the cloud, but data itself. Financial filings, weather satellite images, public transit logs.

    It’s doesn’t matter how the data is served — on a plate or on a stick — it’s not the platter, but the food that matters.