Via Michal Migurski’s delicious feed, I found Disks Have Become Tapes, a fascinating set of observations about why MapReduce is so successful. In short (as observed by Doug Cutting at OSCON last year), MapReduce hits the sweet spot by operating at the transfer rate of disks (growing at 20%/year) rather than seek rate (growing at 5%/year) as relational databases do. It also mentions the growing field of column databases, which we’re watching at Radar (e.g., Michael Stonebraker’s appearance at MoneyTech) as driven by capacity growing faster than transfer and seek. Absolutely fascinating stuff.
tags: hard numbers