Database War Stories #7: Google File System and BigTable

Greg Linden of Findory wrote: “I’ve been enjoying your series on O’Reilly Radar about database war stories at popular startups. I was thinking that it would be fantastic if you could get Jeff Dean or Adam Bosworth at Google to chat a little bit about their database issues. As you probably know, Jeff Dean was involved designing BigTable and the Google File System. Adam Bosworth wrote a much discussed post about the need for better, large scale, distributed databases.”

I followed up with mail to Jeff and Adam. Jeff wrote back briefly about BigTable: “Interesting discussion. I don’t have much to add. I’ve been working with a number of other people here at Google on building a large-scale storage system for structured and semi-structured data called BigTable. It’s designed to scale to hundreds or thousands of machines, and to make it easy to add more machines the system and automatically start taking advantage of those resources without any reconfiguration. We don’t have anything published about it yet, but there’s a public talk about BigTable that I gave at University of
Washington last November available on the web (try some searches for bigtable or view the talk).”

So no new war stories here, but I thought the links were well worth passing along. BigTable sounds a lot like what Ian Wilkes of Second Life is wishing for. What do you think the chances are that Google will release this through Google Code?

More entries in the database war stories series: Second Life, Bloglines and Memeorandum, Flickr, NASA World Wind, Craigslist, O’Reilly Research, Findory and Amazon, Brian Aker of MySQL Responds.

tags: