"Big Data Design and Social Science" entries
The Internet of Things has four big data problems
The IoT and big data are two sides of the same coin; building one without considering the other is a recipe for doom.
The Internet of Things (IoT) has a data problem. Well, four data problems. Walking the halls of CES in Las Vegas last week, it’s abundantly clear that the IoT is hot. Everyone is claiming to be the world’s smartest something. But that sprawl of devices, lacking context, with fragmented user groups, is a huge challenge for the burgeoning industry.
What the IoT needs is data. Big data and the IoT are two sides of the same coin. The IoT collects data from myriad sensors; that data is classified, organized, and used to make automated decisions; and the IoT, in turn, acts on it. It’s precisely this ever-accelerating feedback loop that makes the coin as a whole so compelling.
Nowhere are the IoT’s data problems more obvious than with that darling of the connected tomorrow known as the wearable. Read more…
Becoming data driven
DJ Patil and Hilary Mason's Data Driven: Creating a Data Culture is about building organizations that can take advantage of data.
I’m not going to summarize their work here: you should read it. It’s based on the realization that merely assembling a bunch of people who understand statistics doesn’t do the job. You end up with a group of data specialists on the margins of the organization, who don’t have the ability to do anything more than be frustrated. If you don’t develop a data culture, if people don’t understand the value of data and how it can be used to inform discussions, you can build all the dashboards and Hadoop clusters you want, but they won’t help you.
Data is a powerful tool, but it’s easy to jump on the data bandwagon and miss the benefits. Data Driven: Creating a Data Culture is about building organizations that can really take advantage of data. Is that organization yours? Read more…
Top keynotes at Strata Conference and Strata + Hadoop World 2014
From data privacy to real-world problem solving, O’Reilly’s data editors highlight the best of the best talks from 2014.
2014 was a year of tremendous growth in the field of data, as it was as well for Strata and Strata + Hadoop World, O’Reilly’s and Cloudera’s series of data conferences. At Strata, keynotes, individual sessions, and tracks like Hardcore Data Science, Hadoop and Beyond, Data-Driven Business Day, and Design & Interfaces, among others, explore the cutting-edge aspects of how to gather, store, wrangle, analyze, visualize, and make decisions with the vast amounts of data on our hands today. Looking back on the past year of Strata, the O’Reilly data editors chose our top keynotes from Strata Santa Clara, Strata Barcelona, and Strata + Hadoop World NYC.
It was tough to winnow the list down from an exceptional set of keynotes. Visit the O’Reilly YouTube channel for a larger set of 2014 keynotes, or Safari for videos of the keynotes and many of the conference sessions.
Best of the best
- Julia Angwin reframes the issue of data privacy as justice, due process, and human rights (and her account of trying to buy better privacy goods and services is both instructive and funny).
The big data sweet spot: policy that balances benefits and risks
Deciding what data to collect is hard when consequences are unpredictable.
A big reason why discussions of “big data” get complicated — and policy-makers resort to vague hand-waving, as in the well-known White House executive office report — is that its ripple effects travel fast and far. Your purchase, when recorded by a data broker, affects not only the the ads and deals they offer you in the future, but the ones they offer innumerable people around the country that share some demographic with you.
Policy-making might be simple if data collectors or governments could say, “We’ll collect certain kinds of data for certain purposes and no others” — but the impacts of data collection are rarely predictable. And if one did restrict big data that way, its value would be seriously reduced.
Follow my steps: big data privacy vs collection
Data collection will explode as we learn how to connect you to different places you’ve been by the way you walk or to recognize certain kinds of diseases by your breath.
When such data exhaust is being collected, you can’t evade consequences by paying cash and otherwise living off the grid. In fact, trying to do so may disadvantage you even more: people who lack access to sophisticated technologies leave fewer tracks and therefore may not be served by corporations or governments. Read more…
Big questions, good data, smart designs
Spotify’s Rochelle King on designing with data for optimal user experiences.
Download a free copy of “The New Design Fundamentals” ebook, a curated collection of chapters from our Design library. Rochelle King is the co-author of the forthcoming book “Designing with Data,” which is included in the compilation. King also is co-chairing our newly announced Design Conference — the call for proposals is open; the deadline for submission is July 20, 2015.
The best designers possess empathy for their users — and data is key to gaining this kind of understanding. I recently sat down with Rochelle King, vice president of global design and UX at Spotify, to talk about data, design, and user experience. King stressed the importance of understanding what the numbers in the data mean — and on staying focused on the real endgame:
“I think a common misconception or misstep that happens is people just look at the numbers and they forget that they’re actually representing human behavior. That’s a dangerous cycle you can fall into — you can start to optimize the numbers for the sake of optimizing numbers rather than thinking about building an experience that helps to craft the best experiences possible. You start to focus on improving numbers rather than on making a great experience.”
The benefits of data-informed decision-making are clear, but the process is far from easy. Read more…
Big data’s big ideas
From cognitive augmentation to artificial intelligence, here's a look at the major forces shaping the data world.
Looking back at the evolution of our Strata events, and the data space in general, we marvel at the impressive data applications and tools now being employed by companies in many industries. Data is having an impact on business models and profitability. It’s hard to find a non-trivial application that doesn’t use data in a significant manner. Companies who use data and analytics to drive decision-making continue to outperform their peers.
Up until recently, access to big data tools and techniques required significant expertise. But tools have improved and communities have formed to share best practices. We’re particularly excited about solutions that target new data sets and data types. In an era when the requisite data skill sets cut across traditional disciplines, companies have also started to emphasize the importance of processes, culture, and people. Read more…