FEATURED STORY

Bridging the gap in big data silos

The O'Reilly Radar Podcast: John Carnahan on holistic data analysis, engagement channels, and data science as an art form.

Grain_Storage_Silos_Kool_Cats_Photography_Flickr

In this Radar Podcast episode, I sit down with John Carnahan, executive vice president of data science at Ticketmaster. At our recent Strata + Hadoop World Conference in San Jose, CA, Carnahan presented a session on using data science and machine learning to improve ticket sales and marketing at Ticketmaster.

I took the opportunity to chat with Carnahan about Ticketmaster’s evolving approach to data analysis, the avenues of user engagement they’re investigating, and how his genetics background is informing his work in the big data space.

When Carnahan took the job at Ticketmaster about three years ago, his strategy focused on small, concrete tasks aimed at solving distinct nagging problems: how do you address large numbers of tickets not sold at an event, how do you engage and market those undersold events to fans, and how do you stem abuse of ticket sales. This strategy has evolved, Carnahan explained, to a more holistic approach aimed at bridging the data silos within the company:

“We still want those concrete things, but we want to build a bed of data science assets that’s built on top of a company that’s been around almost 40 years and has a lot of data assets. How do we build the platform that will leverage those things into the future, beyond just those small niche products that we really want to build. We’re trying to bridge the gap between a lot of those products, too. Rather than think of each of those things as a vertical or a silo that’s trying to accomplish something, it’s how do you use something that you’ve built over here, over there to make that better?”

Read more…

Comment
Four short links: 5 March 2015

Four short links: 5 March 2015

Web Grain, Cognition and Computation, New Smart Watch, and Assessing Accuracy

  1. The Web’s Grain (Frank Chimero) — What would happen if we stopped treating the web like a blank canvas to paint on, and instead like a material to build with?
  2. Bruce Sterling on Convergence of Humans and MachinesI like to use the terms “cognition” and “computation”. Cognition is something that happens in brains, physical, biological brains. Computation is a thing that happens with software strings on electronic tracks that are inscribed out of silicon and put on fibre board. They are not the same thing, and saying that makes the same mistake as in earlier times, when people said that human thought was like a steam engine.
  3. Smart Pocket Watch — I love to see people trying different design experiences. This is beautiful. And built on Firefox OS!
  4. Knowledge-Based Trust (PDF) — Google research paper on how to assess factual accuracy of web page content. It was bad enough when Google incentivised people to make content-free pages. Next there’ll be a reward for scamming bogus facts into Google’s facts database.
Comment

Empathy is a state of mind, not a specific technique

A design process paved with empathic observations will lead you, slowly and iteratively, to a better product.

Editor’s note: this post was originally published on the author’s blog, Exploring the world beyond mobile; this lightly edited version is republished here with permission.

steel_framing_Julien_Belli_Flickr

If I’m ever asked what’s most important in UX design, I always reply “empathy.” It’s the core meta attribute, the driver that motivates everything else. Empathy encourages you to understand who uses your product, forces you to ask deeper questions, and motivates the many redesigns you go through to get a product right.

But empathy is a vague concept that isn’t strongly appreciated by others. There have been times when talking to product managers that my empathy-driven fix-it list will get a response like, “We appreciate that Scott, but we have so much to get done on the product, we don’t have time to tweak things like that right now.” Never do you feel so put in your place when someone says that your job is “tweaking.”

The paradox of empathy is that while it drives us at a very deep level, and ultimately leads us to big, important insights, it usually starts small. The empathic process typically notices simple things like ineffective error messages, observed user workarounds, or overly complicated dialog boxes. Empathy starts with very modest steps. However, these small observations are the wedge that splits the log; it’s these initial insights, if you follow them far enough, that open up your mind and lead you to great products.

Read more…

Comment

7 user research myths and mistakes

Finding the holes in qualitative and quantitative testing.

testing
I can’t tell you how often I hear things from engineers like, “Oh, we don’t have to do user testing. We’ve got metrics.” Of course, you can almost forgive them when the designers are busy saying things like, “Why would we A/B test this new design? We know it’s better!”

In the debate over whether to use qualitative or quantitative research methods, there is plenty of wrong to go around. So, let’s look at some of the myths surrounding qualitative and quantitative research, and the most common mistakes people make when trying to use them.
Read more…

Comment

Year Zero: Our life timelines begin

In the next decade, Year Zero will be how big data reaches everyone and will fundamentally change how we live.

sneaky_peek_Heart_of_Oak

Editor’s note: this post originally appeared on the author’s blog, Solve for Interesting. This lightly edited version is reprinted here with permission.

In 10 years, every human connected to the Internet will have a timeline. It will contain everything we’ve done since we started recording, and it will be the primary tool with which we administer our lives. This will fundamentally change how we live, love, work, and play. And we’ll look back at the time before our feed started — before Year Zero — as a huge, unknowable black hole.

This timeline — beginning for newborns at Year Zero — will be so intrinsic to life that it will quickly be taken for granted. Those without a timeline will be at a huge disadvantage. Those with a good one will have the tricks of a modern mentalist: perfect recall, suggestions for how to curry favor, ease maintaining friendships and influencing strangers, unthinkably higher Dunbar numbers — now, every interaction has a history.

This isn’t just about lifelogging health data, like your Fitbit or Jawbone. It isn’t about financial data, like Mint. It isn’t just your social graph or photo feed. It isn’t about commuting data like Waze or Maps. It’s about all of these, together, along with the tools and user interfaces and agents to make sense of it.

Every decade or so, something from military or enterprise technology finds its way, bent and twisted, into the mass market. The client-server computer gave us the PC; wide-area networks gave us the consumer web; pagers and cell phones gave us mobile devices. In the next decade, Year Zero will be how big data reaches everyone. Read more…

Comment: 1

What experts do: Curse of the intermediate

A framework for what separates those whose skills continue to build and those who stall out no matter how much they try.

We all know the Curse of Expertise — that thing that makes most experts awful at imagining what it’s like to be a novice. The Curse of Expertise makes tech editors weep and readers seethe. “Where’s the empathy?!” we say, as if the expert had a conscious choice. But they mostly don’t. The Curse of Expertise is not a problem for which MOAR EMPATHY is the solution. Experts don’t lack empathy; they lack the security clearance to the part of their brain where their cognitive biases live. Subconscious cognitive biases. And those biases don’t just make us (experts) fail at predicting the struggle of novices, they can also make us less likely to see novel solutions to well-worn problems.

But given a choice to suddenly be an expert or a novice, we’d pick Curse of the Sucks-To-Be-Me Expert over Curse of the I-Suck-At-This Novice. There’s a third curse, though. The mastery curve is, of course, not binary, but a continuum from first-time to Jiro-Dreams-Of-Sushi. And there in the middle? The Curse of the Intermediate. The Curse of the Intermediate is the worst because it’s the place where hopes and dreams of expertise go to die. The place where even the most patient practicer eventually believes they just don’t have what it takes. Read more…

Comment