Information Commons

Harbor Research released a white paper discussing MAYA's Information Commons and MIT's Internet 0. Information Commons interested me when I heard about it earlier this year, and I was glad to feature it at Where 2.0. I didn't get to tease out the cool technology underneath it, though, which the white paper begins to do.

The Information Commons is built on a p2p database system. It's a visualization environment for navigating data and their relationships. For more detail, check this paper by the MAYA founders. The data organization is quite interesting: each data item is given a UUID and is expressed as key-value pairs. The data aren't typed when they're stored, but instead when they're read. It reminds me a little of the power of the URL, a little of a system that Rael built recently, and a little of Cal Henderson's talk about Flickr where he famously said that "normalized data is for sissies". There's a lot of thought lately going into breaking out of the traditional relational model, and not just in the direction of OO databases.

Now every time you talk about going outside the relational model, you get yelled at by the relational people. I can see people who are perfectly happy with MySQL, PostgreSQL, and SQL*Server, queueing to yell at me already for daring to suggest something as foolish. I think of it as complementary. Every database system is designed by weighing different factors. Relational databases are great low-level systems for efficiently storing and querying databases, but they suck at higher-level queries. Anyone who has ever debugged an SQL statement spanning more than three tables and taking more than five lines on the screen knows what I'm talking about. Hence the huge object-relational mapping energy lately (of which ActiveRecord most rocks my world).

The MAYA database system is designed by looking at the problem of identifying related information in different databases. In a sense, it's a loose federation system (with migratory caching and all sorts of other stuff built on top) for databases, solving problems that you don't encounter when you implement a single database, but the ones you encounter when you try to deal with dozens of different ones from different organizations. I think the hyping of it as "the web's replacement" is froth, but the underlying technology reflects an interesting approach to solving an interesting problem. Ziff Davis covered the Information Commons and the whitepaper in this blog entry. (updated Oct 3 to reflect the all-caps nature of the company name MAYA)