As society becomes increasingly data driven, it's critical to remember big data isn't a magical tool for predicting the future.
If you eat ice cream, you’re more likely to drown.
That’s not true, of course. It’s just that both ice cream and swimming happen in the summer. The two are correlated — and ice cream consumption is a good predictor of drowning fatalities — but ice cream hardly causes drowning.
These kinds of correlations are all around us, and big data…
Eleven areas of focus for deeper investigation.
Conferences like Strata are planned a year in advance. The logistics and coordination required for an event of this magnitude takes a lot of planning, but it also takes a decent amount of prediction: Strata needs to skate to where the puck is going.
While Strata New York + Hadoop World 2013 is still a few months away, we’re…
The cycle of good, bad, and stable has happened at every layer of the stack. It will happen with big data, too.
First, technology is good. Then it gets bad. Then it gets stable.
This has been going on for a long time, likely since the invention of fire, knives, or the printed word. But I want to focus specifically on computing technology. The human race is busy colonizing a second online world and sticking prosthetic brains — today, we call them…
We need checks and balances to ensure data-driven predictions don't become prejudices.
A couple of years ago, I spoke with a European Union diplomat who shall remain nameless about the governing body’s attitude toward privacy.
“Do you know why the French hate traffic…
The biggest threat that a data-driven world presents is an ethical one.
Since the first of our ancestors chipped stone into weapon, technology has divided us. Seldom more than today, however: a connected, always-on society promises health, wisdom, and efficiency even as it threatens an end to privacy and the rise of prejudice masked as science.
On its surface, a data-driven society is more transparent, and makes better uses of its resources….
Further reading and discussion on the civil rights implications of big data.
A few weeks ago, I wrote a post about big data and civil rights, which seems to have hit a nerve. It was posted on Solve for Interesting and here on Radar, and then folks like Boing Boing picked it up.
I haven’t had this kind of response to a post before (well, I’ve had responses, such…
Looking ahead at big data's role in enterprise business intelligence, civil engineering, and customer relationship optimization.
In the past couple of years, marketers and pundits have spent a lot of time labeling everything ”big data.” The reasoning goes something like this:
Everything is on the Internet.
The Internet has a lot of data.
Therefore, everything is big data.
What the data is must be linked to how it can be used.
Data doesn’t invade people’s lives. Lack of control over how it’s used does.
What’s really driving so-called big data isn’t the volume of information. It turns out big data doesn’t have to be all that big. Rather, it’s about a reconsideration of the fundamental economics of analyzing data.
For decades, there’s been a fundamental tension between three attributes of databases. You…
A glimpse into enterprise use of big data.
Feedback from a recent Strata Online Conference suggests there's a large demand for clear information on what big data is and how it will change business.
Companies that employ data feedback loops are poised to dominate their industries.
We're moving beyond an information economy. The efficiencies and optimizations that come from constant and iterative feedback will soon become the norm for businesses and governments.