Here are a few stories from the data space that caught my attention this week.
Big data, big politics
In the aftermath of the US presidential election, much attention has been focused on Nate Silver’s art of predicting the election results with data. Some looked at it from a coverage angle and how Silver’s work in the spotlight will affect the process of covering elections in the future. John McDermott reports at AdAge that Silver’s work will help shift the “nebulous aspects” of reporting that focus on “feel” and “momentum” to reporting that is anchored in facts and statistics. ComScore analyst Andrew Lipsman said to McDermott, “Now that people have seen [statistics-driven political analysis] proven over a couple of cycles, people will be more grounded in the numbers.”
Which also shows the attention Silver attracted may serve to help democratize big data as well. Tarun Wadhwa reports at Forbes that the power of big data has finally been realized in the US political process:
“Beyond just personal vindication, Silver has proven to the public the power of Big Data in transforming our electoral process. We already rely on statistical models to do everything from flying our airplanes to predicting the weather. This serves as yet another example of computers showing their ability to be better at handling the unknown than loud-talking experts. By winning ‘the nerdiest election in the history of the American Republic,’ Barack Obama has cemented the role of Big Data in every aspect of the campaigning process. His ultimate success came from the work of historic get-out-the-vote efforts dominated by targeted messaging and digital behavioral tracking.”
Michael Scherer at Time has an in-depth look at the role big data and data mining played in Obama’s campaign as well. Campaign manager Jim Messina, Scherer writes, “promised a totally different, metric-driven kind of campaign in which politics was the goal but political instincts might not be the means” and hired dozens of data crunchers to establish an analytics department. The team put together a massive database that merged information from all areas of the campaign — social media, pollsters, consumer databases, fundraisers, etc. — into one central location. Scherer reports: “The new megafile didn’t just tell the campaign how to find voters and get their attention; it also allowed the number crunchers to run tests predicting which types of people would be persuaded by certain kinds of appeals.”
Scherer’s piece is a fascinating look at how data was put to use in a successful presidential campaign. It is this week’s recommended read.
Big data trends, and putting personal big data up for sale
Derrick Harris writes this week at GigaOm that “big data has turned from a buzzword and concept best left for large web companies into a force that drives much of our digital lives.” He takes a look at five trends that will shape big data’s evolution in the near future. He points to the growing demand for — and investment in — data scientists; Hadoop’s evolving platform; machine learning; and how big data is being turned into applications, products, and services. He also looks at the role mobile data is playing in AI, noting that our smartphones are rich sources of personal data, and as technology improves, the devices are becoming more and more involved in our daily lives. Harris writes:
“[Our phones] know where we go, who our friends are, what’s on our calendars and what we look at online. Thanks to a new generation of applications such as Siri, Saga and Google Now trying to serve as personal assistants, our phones can understand what we say, know the businesses we frequent and the foods we eat, and the hours we’re at home, at work or out on the town. Already, their developers claim such apps can augment our limited vantage point by automatically telling us the best directions to our upcoming appointment, or the best place to get our favorite foods in a city the app knows we haven’t been to before.”
In a related post, Harris looks at the collection of all that data, and who has power over it and profits from it. Noting the shift toward consumer power with functions such as “Do Not Track” and potential privacy legislation, Harris looks at startups and incentive programs that are betting people will be a lot more open with their data — and more open with more valuable data — if they stand to make money off it.
Harris highlights Enliken, a data broker startup that’s looking to facilitate exchanges between consumers and advertisers. The company starts by showing consumers exactly what kind of personal data is already being gathered, better informing them what they might be able to sell instead of clicking the “Do Not Track” button. “With Enliken,” Harris writes, “users can edit, control and even prioritize the data they want advertisers to see, thus making the data richer and more-valuable.” He also looks at company’s like Google that are looking for a direct sale, offering consumers gift cards in exchange for tracking permission. He writes that the model may have traction — as long as consumers don’t have to do much work, he says, they seem perfectly willing to share data in exchange for, say, a $5 gift card to Target.
Cloud-user rights at stake
The Megaupload case has been getting quite a bit of ink over recent weeks, mainly focused on the pending fate of Megaupload founder Kim Dotcom. In a shift from the regular coverage, David Kravets writes at Wired this week that the ramifications of this case reach beyond criminal prosecution. He points to a lawsuit filed by the Electronic Frontier Foundation on behalf of Megaupload user Kyle Goodwin who has been denied access to his data since the US federal government shut down the service last January, noting the outcome of the lawsuit could set a precedent not only for other Megaupload users, but for cloud user rights in general. Kravets reports:
“.. even if a system is put in place for users to get back their files, it’s likely the data would first need to be reviewed by the government or a third party to determine if any of the data infringed copyrights, says EFF attorney Julie Samuels, because the government would oppose returning such data to account holders. But this raises serious issues about property rights and privacy at a time when more and more people are using the cloud.”
Kravets writes that “[s]o far, federal prosecutors are proposing a process that would make it essentially impossible for former Megaupload users to recover any of their legitimate data.”
Tip us off
News tips and suggestions are always welcome, so please send them along.