Such lists might mean we miss the truly great breakthroughs, inspirations, and leaps of faith necessary to evolve.
Editor’s note: this post originally appeared on Tilt the Windmill; it is republished here with permission.
First: it’s an excellent post. You should read it. I’ll wait.
Every enterprise decision-maker will soon be running their business according to the lists Barry envisions, as the power of big data and analytics finds its way into every boardroom and dashboard. Society will soon demand them, too. But while such analysis is tremendously valuable, it carries two dangers: the politics of setting criteria, and the trap of relying on data for inspiration.
The harsh light of data
Barry is right: rather than using our precious time and resources to make yet another linkbait list of the 50 cutest kittens, or the seven people I’ll try to avoid at SXSW, we should use abundant data and a connected world to build lists that matter: lying politicians, bad cars, lousy doctors. Then we can use these lists to change policy and behaviour because we’ll make things transparent. Shining the harsh light of data on something can improve it.
As society becomes increasingly data driven, it's critical to remember big data isn't a magical tool for predicting the future.
If you eat ice cream, you’re more likely to drown.
That’s not true, of course. It’s just that both ice cream and swimming happen in the summer. The two are correlated — and ice cream consumption is a good predictor of drowning fatalities — but ice cream hardly causes drowning.
These kinds of correlations are all around us, and big data makes them easy to find. We can correlate childhood trauma with obesity, nutrition with crime rates, and how toddlers play with future political affiliations.
Just as we wouldn’t ban ice cream in the hopes of preventing drowning, we wouldn’t preemptively arrest someone because their diet wasn’t healthy. But a quantified society, awash in data, might be tempted to do so because overwhelming correlation looks a lot like causality. And overwhelming correlation is what big data does best.
It’s getting easier than ever to find correlations. Parallel computing, advances in algorithms, and the inexorable crawl of Moore’s Law have dramatically reduced how much it costs to analyze a data set. Consider an activity we do dozens of times a day, without thinking: a Google search. The search is farmed out to thousands of machines, and often returns hundreds of answers in less than a second. Big data might seem esoteric, but it’s already here. Read more…
Eleven areas of focus for deeper investigation.
Conferences like Strata are planned a year in advance. The logistics and coordination required for an event of this magnitude takes a lot of planning, but it also takes a decent amount of prediction: Strata needs to skate to where the puck is going.
While Strata New York + Hadoop World 2013 is still a few months away, we’re already guessing at what next year’s Santa Clara event will hold. Recently, the team got together to identify some of the hot topics in big data, ubiquitous computing, and new interfaces. We selected eleven big topics for deeper investigation.
- Deep learning
- Time-series data
- The big data “app stack”
- Cultural barriers to change
- Design patterns
- Laggards and Luddites
- The convergence of two databases
- The other stacks
- Mobile data
- The analytic life-cycle
- Data anthropology
Here’s a bit more detail on each of them. Read more…
It’s been a weird couple of weeks for the Internet of Things. As we connect everything to everything else, we inadvertently create a huge attack surface for hackers, and we’re starting to see the chinks in the armor.
Let’s say you fancy a fast car. Flavio Garcia, a University of Birmingham computer scientist, discovered the algorithim that verifies the ignition key for luxury cars like Porsches, Audis, Bentleys, and Lamborghinis. He was slapped with an injunction to ban him from disclosing his findings at the Usenix Security Symposium in order to prevent sophisticated criminal gangs from having the analytics tools for widespread car theft.
You might need Garcia’s algorithm to steal a car, but soon, with an entirely different algorithm, you may be able to crash one into a tree or disable its brakes from a distance. Or maybe it’s a fast boat you’re after. Mess with its GPS, and you can steer it where you want without the crew noticing.
Learn to resist vanity metrics
One of the things we preach in Lean Analytics is that entrepreneurs should avoid vanity metrics—numbers that make you feel good, but ultimately, don’t change your behavior. Vanity metrics (such as “total visitors”) tend to go “up and to the right” but don’t tell you much about how you’re doing.
Many people find solace in graphs that go up and to the right. The metric “Total number of people who have visited my restaurant” will always increase; but on its own it doesn’t tell you anything about the health of the business. It’s just head-in-the-sand comforting.
A good metric is often a comparative rate or ratio. Consider what happens when you put the word “per” before or after a metric. “Restaurant visitors per day” is vastly more meaningful. Time is the universal denominator, since the universe moves inexorably forwards. But there are plenty of other good ratios. For example, “revenue per restaurant visitor” matters a lot, since it tells you what each diner contributes.
What’s an active user, anyway?
For many businesses, the go-to metric revolves around “active users.” In a mobile app or software-as-a-service business, only some percentage of people are actively engaged. In a media site, only some percentage uses the site each day. And in a loyalty-focused e-commerce company, only some buyers are active.
This is true of more traditional businesses, too. Only a percentage of citizens are actively engaged in local government; only a certain number of employees are using the Intranet; only a percentage of coffee shop patrons return daily.
Unfortunately, saying “measure active users” begs the question: What’s active, anyway?
To figure this out, you need to look at your business model. Not your business plan, which is a hypothetical projection of how you’ll fare, but your business model. If you’re running a lemonade stand, your business model likely has a few key assumptions:
- The cost of lemonade;
- The amount of foot traffic past your stand;
- The percent of passers-by who will buy from you;
- The price they are willing to pay.
Our Lean lemonade stand would then set about testing and improving each metric, running experiments to find the best street corner, or determine the optimal price.
Lemonade stands are wonderfully simple, so your business may have many other assumptions, but it is essential that you quantify them and state them so you can then focus on improving them, one by one, until your business model and reality align. In a restaurant, for example, these assumptions might be, “we will have at least 50 diners a day” or “diners will spend on average $20 a meal.”
The activity you want changes
We believe most new companies and products go through five distinct stages of growth:
- Empathy, where you figure out what problem you’re solving and what solution people want;
- Stickiness, where you measure how many people adopt your solution rather than trying it and leaving;
- Virality, where you maximize word-of-mouth and references;
- Revenue, where you pour some part of your revenues back into paid acquisition or advertising;
- Scale, where you grow the business through automation, delegation, and process.
Submit your suggestions for videos that make us think about how data, visualizations, and technology are changing us
Each year at Strata, we warm up the crowd in the main keynote sessions with short videos that will make people think. These videos demonstrate the ways that data, technology, and visualization are changing us. Some are funny; some are clever; some are downright disturbing.
For Strata New York + Hadoop World in October, we’re hoping you’ll join in and suggest some videos for us. If you’ve got something you feel captures the zeitgeist of technology at the fringes, then complete this form, and we’ll check it out. We’ll choose some of them as we kick off the event this fall.
Healthy changes that fit into a busy schedule.
The last three years haven’t been very healthy. In addition to raising a new daughter, I’ve been launching Strata and Startupfest and working with Ben Yoskovitz on Lean Analytics. It’s been rewarding, and fun, but it hasn’t been good for my waistline. I borrowed a joke from Emo Phillips last week at an event in Toronto: my body isn’t a temple; at best, it’s a poorly maintained Presbyterian youth center.
Nilofer Merchant calls sitting “the smoking of our generation,” and that’s not hyperbole. Lured into chairs by our online lives, we’ve become sedentary. Our children are growing, horizontally, at an alarming rate. And when we do get up, it’s often to sit elsewhere — over lunch, in a coffee shop, and so on.
In a series of conversations over the last few weeks, Nilofer and I have been discussing all manner of things, from the power of networks to how to change behavior. Her admonishment to get out and walk got me looking for other simple hacks that might help me be healthier.
We asked the Startup Showcase judges three questions about the big data industry.
The Startup Showcase returns to Strata this month, with 10 startup finalists pitching our panel of judges. We’ve assembled an enviable— and somewhat intimidating— lineup of experts to help narrow down the field.
In the interest of giving our finalists a head start, we asked the judges three questions about the big data industry.
Design compels. Math is proof. Both sides will defend their domains at Strata's next Great Debate.
At Strata Santa Clara later this month, we’re reprising what has become a tradition: Great Debates. These Oxford-style debates pit two teams against one another to argue a hot topic in the fields of big data, ubiquitous computing, and emerging interfaces.
Part of the fun is the scoring: attendees vote on whether they agree with the proposal before the debaters; and after both sides have said their piece, the audience votes again. Whoever moves the needle wins.
This year’s proposition — that design matters more than math — is sure to inspire some vigorous discussion. The argument for math is pretty strong. Math is proof. Given enough data — and today, we have plenty — we can know. “The right information in the right place just changes your life,” said Stewart Brand. Properly harnessed, the power of data analysis and modeling can fix cities, predict epidemics, and revitalize education. Abused, it can invade our lives, undermine economies, and steal elections. Surely the algorithms of big data matter!
But your life won’t change by itself. Bruce Mau defines design as “the human capacity to plan and produce desired outcomes.” Math informs; design compels. Without design, math can’t do its thing. Poorly designed experiments collect the wrong data. And if the data can’t be understood and acted upon, it may as well not have been crunched in the first place.
This is the question we’ll be putting to our debaters: Which matters more? A well-designed collection of flawed information — or an opaque, hard-to-parse, but unerringly accurate model? From mobile handsets to social policy, we need both good math and good design. Which is more critical? Read more…
How the inevitable rise of software means cycle time trumps scale.
Exponential curves gradually, inexorably grow until they reach a limit. The function increases over time. That’s why a force like gravity, which grows exponentially as objects with mass get closer to one another, eventually leads to a black hole. And at the middle of this black hole is a point of infinite mass, a singularity, within which the rules no longer apply.
Financiers also like exponents. “Compound interest is the most powerful force in the universe” is a quote often attributed to Einstein; whoever said it was right. If you pump the proceeds of interest back into a bank account, it’ll increase steadily.
Computer scientists like to throw the term “singularity” around, too. To them, it’s the moment when machines become intelligent enough to make a better machine. It’s the Geek Rapture, the capital-S-Singularity. It’s the day when machines don’t need us any more, and to them, we look like little more than ants. Ray Kurzweil thinks it’s right around the corner — circa 2045 — and after that time, to us, these artificial intelligences will be incomprehensible.
Businesses need to understand singularities, because they have one of their own to contend with. Read more…