FEATURED STORY

The science of moving dots: the O’Reilly Data Show Podcast

Rajiv Maheswaran talks about the tools and techniques required to analyze new kinds of sports data.

Editor’s note: you can subscribe to the O’Reilly Data Show Podcast through SoundCloud or through our RSS feed.

Many data scientists are comfortable working with structured operational data and unstructured text. Newer techniques like deep learning have opened up data types like images, video, and audio.

Other common data sources are garnering attention. With the rise of mobile phones equipped with GPS, I’m meeting many more data scientists at start-ups and large companies who specialize in spatio-temporal pattern recognition. Analyzing “moving dots” requires specialized tools and techniques. A few months ago, I sat down with Rajiv Maheswaran founder and CEO of Second Spectrum, a company that applies analytics to sports tracking data. Maheswaran talked about this new kind of data and the challenge of finding patterns:

“It’s interesting because it’s a new type of data problem. Everybody knows that big data machine learning has done a lot of stuff in structured data, in photos, in translation for language, but moving dots is a very new kind of data where you haven’t figured out the right feature set to be able to find patterns from. There’s no language of moving dots, at least not that computers understand. People understand it very well, but there’s no computational language of moving dots that are interacting. We wanted to build that up, mostly because data about moving dots is very, very new. It’s only in the last five years, between phones and GPS and new tracking technologies, that moving data has actually emerged.”

Read more…

Comments: 2

Uber is breaking bad

Uber has built a great service. Why do they feel the need to use dirty tricks to succeed?

Light_artifacts_sfullenwider_Flickr

Tim O’Reilly has said that Uber is an example of designing for how the world ought to be. Their app works well, their cars are clean, their drivers are pleasant, and they usually arrive quickly. But more goes into the experience of a company than just an app. Corporate behavior is also part of the company’s design; perhaps not as noticeable as their Android or iPhone app, but a very real part. That’s where Uber falls down. They have increasingly been a bad actor, on many counts:

I could go on (advertising hot female drivers, abuses of their privacy policy, and more), but I won’t. You get the point. This is #GamerGate, but with a $17 billion valuation behind it. Read more…

Comment: 1
Four short links: 20 November 2014

Four short links: 20 November 2014

Postmortems, Cloud Triggers, IoT Desires, and Barbie Can Code

  1. The Infinite Hows (John Allspaw) — when finding ways to improve systems to prevent errors, the process of diagnosis should be focused on the systems and less on the people. (aka “human error” is the result of a preceding systems error.) (aka “design for failure.”)
  2. Amazon Lambda — triggers in the cloud.
  3. Enchanted Objects (PNG) — organizing the Internet of Thing by human desires. (via Designing the Enchanted Future)
  4. Barbie Remixed (PDF) — brilliant remix of a book that missed the mark into one that hits the bullseye.
Comment

Safari for Schools: Free O’Reilly material for K-12

Safari is offering O’Reilly books and videos for free to every K-12 student and teacher in the U.S.

This past February, Tim O’Reilly brought me into an email thread with the White House with a straightforward but urgent request — could Safari provide the delivery mechanism to make all of O’Reilly Media’s titles available to every K–12 student in America? Commitments to the President’s “ConnectED” program were lined up from a number of software, hardware, and networking companies, but connected devices would be much more useful with content included. We’re proud that we were able to say yes to something so important — and on such short notice.

It made sense for Safari to deliver on O’Reilly’s commitment, as our business is providing online access to thousands of the best books and training courses to companies and organizations of all sizes. But as we started unpacking the particulars, we uncovered more complexity than we expected. For example, there are tens of thousands of school districts across the country, each with their own IT infrastructure. It simply wouldn’t scale if providing access to every student also meant working directly with every school or district. Compliance with a set of regulations designed to protect children’s privacy (known as COPPA) meant that we couldn’t simply open up our standard platform to students.

Constraints can be wonderful in focusing attention, and fortunately the outstanding team at Safari was up for the challenge. By September 1, we had quietly opened up a beta site where any high school student could apply for access to the full collection of O’Reilly books and videos.

In conjunction with today’s White House event promoting “Future Ready Schools,” I’m thrilled to say that we have delivered on the pledge to make the full catalog of O’Reilly books and videos available for free to any K–12 student in America, more than a month ahead of our original January 2015 promise.

Read more…

Comment

The intersection of data and design is equal parts art and science

Data-informed design is a framework to hone understanding of customer behavior and align teams with larger business goals.

Editor’s note: this is an excerpt from our forthcoming book Designing with Data; it is part of a free curated collection of chapters from the O’Reilly Design library — download a free copy of the Experience Design ebook here.

The phrase “data driven” has long been part of buzzword-bingo card sets. It’s been heard in the halls of the web analytics conference eMetrics for more than a decade, with countless sessions aimed at teaching audience members how to turn their organizations into data-driven businesses.

When spoken of in a positive light, the phrase data driven conjures visions of organizations with endless streams of silver-bullet reports — you know the ones: they’re generally entitled something to the effect of “This Chart Will Help Us Fix Everything” and show how a surprise change can lead to a quadrillion increase in revenue along with world peace.

When spoken of in a negative light, the term is thrown around as a descriptor of Orwellian organizations with panopticon-level data collection methods, with management imprisoned by relentless reporting, leaving no room for real innovation.

Evan Williams, founder of Blogger, Twitter, and Medium, made an apt comment about being data driven:

I see this mentality that I think is common, especially in Silicon Valley with engineer-driven start-ups who think they can test their way to success. They don’t acknowledge the dip. And with really hard problems, you don’t see market success right away. You have to be willing to go through the dark forest and believe that there’s something down there worth fighting the dragons for, because if you don’t, you’ll never do anything good. I think it’s kind of problematic how data-driven some companies are today, as crazy as that sounds.”

Read more…

Comment

Introducing “A Field Guide to the Distributed Development Stack”

Tools to develop massively distributed applications.

Editor’s Note: At the Velocity Conference in Barcelona we launched “A Field Guide to the Distributed Development Stack.” Early response has been encouraging, with reactions ranging from “If I only had this two years ago” to “I want to give a copy of this to everyone on my team.” Below, Andrew Odewahn explains how the Guide came to be and where it goes from here.


As we developed Atlas, O’Reilly’s next-generation publishing tool, it seemed like every day we were finding interesting new tools in the DevOps space, so I started a “Sticky” for the most interesting-looking tools to explore.

field-guide-sticky

At first, this worked fine. I was content to simply keep a list, where my only ordering criteria was “Huh, that looks cool. Someday when I have time, I’ll take a look at that,” in the same way you might buy an exercise DVD and then only occasionally pull it out and think “Huh, someday I’ll get to that.” But, as anyone who has watched DevOps for any length of time can tell you, it’s a space bursting with interesting and exciting new tools, so my list and guilt quickly got out of hand.

Read more…

Comment