Mac Slocum

Mac Slocum is director of online content at O'Reilly. He's been writing, editing and producing web content in various forms since the mid '90s. He also dabbles in video interviews from time to time.

Curiosity turned loose on GitHub data

Ilya Grigorik's GitHub project shows what happens when questions, data, and tools converge.

GitHub Archive logoI’m fascinated by people who:

1. Ask the question, “I wonder what happens if I do this?” and then follow it all the way through.

2. Start a project on a whim and open it up so anyone can participate.

Ilya Grigorik (@igrigorik) did both of these things, which is why our recent conversation at Strata Conference + Hadoop World was one of my favorite parts of the event.

By day, Grigorik is a developer advocate on Google’s Make the Web Fast team (he’s a perfect candidate for a future Velocity interview). On the side, he likes to track open source projects on GitHub. As he explained during our chat, this can be a time-intensive hobby:

“I follow about 3,000 open source projects, and I try to keep up with what’s going on, what are people contributing to, what are the new interesting sub-branches of work being done … The problem I ran into about six months ago was that, frankly, it was just too much to keep up with. The GitHub timeline was actually overflowing. In order to keep up, I would have to go in every four hours and scan through everything, and then repeat it. That doesn’t give you much time for sleep.” [Discussed 15 seconds into the interview.]

Grigorik built a system — including a newsletter— that lets him stay in the loop efficiently. He worked with GitHub to archive public GitHub activity, and he then made that data available in raw form and through Google BigQuery (the data is updated hourly).

This is a fun project, no doubt, but it’s also a big deal. Here’s why: When you shorten the distance between questions and answers, you empower people to ask more questions. It’s the liberation of curiosity, and that’s exactly what happened here. Read more…

Comment: 1

Now available: Big Data Now 2012 Edition

O'Reilly's annual data anthology explores the maturation of big data and data science.

Big Data Now 2012 EditionIn the first edition of our free Big Data Now anthology, the O’Reilly team tracked the birth and early development of data tools and data science. Now, with the second edition, we’re seeing what happens when big data grows up: how it’s being applied, where it’s playing a role, and the consequences — good and bad alike — of data’s ascendance.

We’ve organized the 2012 edition of Big Data Now into five areas:

Getting Up to Speed With Big Data — Essential information on the structures and definitions of big data.

Big Data Tools, Techniques, and Strategies — Expert guidance for turning big data theories into big data products.

The Application of Big Data — Examples of big data in action, including a look at the downside of data.

What to Watch for in Big Data — Thoughts on how big data will evolve and the role it will play across industries and domains.

Big Data and Health Care — A special section exploring the possibilities that arise when data and health care come together.

You can download free editions of Big Data Now 2012 in PDF, Mobi and EPUB formats here. The 2011 edition is also available.

Comment

Strata Rx is a wrap

Watch live keynotes from this week's Strata Rx Conference in San Francisco.

The intersection of big data and health care was explored at the O’Reilly Strata Rx Conference. The event has concluded, but you can still access an archive of videos, photos, and speaker slides. Read more…

Comment

Live from the O’Reilly Strata Conference in London

Catch live keynotes from this week's Strata Conference in London.

Experts from across the data world are coming together at the O’Reilly Strata Conference in London this week. You can watch live keynotes from the event below (full broadcast schedule is available here).

Comment

When data disrupts health care

The convergence of data, privacy and cost have created a unique opportunity to reshape health care.

Health care appears immune to disruption. It’s a space where the stakes are high, the incumbents are entrenched, and lessons from other industries don’t always apply.

Yet, in a recent conversation between Tim O’Reilly and Roger Magoulas it became evident that we’re approaching an unparalleled opportunity for health care change. O’Reilly and Magoulas explained how the convergence of data access, changing perspectives on privacy, and the enormous expense of care are pushing the health space toward disruption.

As always, the primary catalyst is money. The United States is facing what Magoulas called an “existential crisis in health care costs” [discussed at the 3:43 mark]. Everyone can see that the current model is unsustainable. It simply doesn’t scale. And that means we’ve arrived at a place where party lines are irrelevant and tough solutions are the only options.

“Who is it that said change happens when the pain of not changing is greater than the pain of changing?” O’Reilly asked. “We’re now reaching that point.” [3:55]

(Note: The source of that quote is hard to pin down, but the sentiment certainly applies.)

This willingness to change is shifting perspectives on health data. Some patients are making their personal data available so they and others can benefit. Magoulas noted that even health companies, which have long guarded their data, are warming to collaboration.

At the same time there’s a growing understanding that health data must be contextualized. Simply having genomic information and patient histories isn’t good enough. True insight — the kind that can improve quality of life — is only possible when datasets are combined.

Read more…

Comments: 4

When data disrupts health care

The convergence of data, privacy and cost have created a unique opportunity to reshape health care.

Health care appears immune to disruption. It’s a space where the stakes are high, the incumbents are entrenched, and lessons from other industries don’t always apply.

Yet, in a recent conversation between Tim O’Reilly and Roger Magoulas it became evident that we’re approaching an unparalleled opportunity for health care change. O’Reilly and Magoulas explained how the convergence of data access, changing perspectives on privacy, and the enormous expense of care are pushing the health space toward disruption.

As always, the primary catalyst is money. The United States is facing what Magoulas called an “existential crisis in health care costs” [discussed at the 3:43 mark]. Everyone can see that the current model is unsustainable. It simply doesn’t scale. And that means we’ve arrived at a place where party lines are irrelevant and tough solutions are the only options.

“Who is it that said change happens when the pain of not changing is greater than the pain of changing?” O’Reilly asked. “We’re now reaching that point.” [3:55]

(Note: The source of that quote is hard to pin down, but the sentiment certainly applies.)

This willingness to change is shifting perspectives on health data. Some patients are making their personal data available so they and others can benefit. Magoulas noted that even health companies, which have long guarded their data, are warming to collaboration.

At the same time there’s a growing understanding that health data must be contextualized. Simply having genomic information and patient histories isn’t good enough. True insight — the kind that can improve quality of life — is only possible when datasets are combined.

Read more…

Comment
“It’s impossible for me to die”

“It’s impossible for me to die”

Julien Smith on the realities of modern safety and our misfiring flinch reflexes.

Julien Smith believes I won’t let him die.

The subject came up during our interview at Foo Camp 2012 — part of our ongoing foo interview series — in which Smith argued that our brains and innate responses don’t always map to the safety of our modern world:

“We’re in a place where it’s fundamentally almost impossible to die. I could literally — there’s a table in front of me made of glass — I could throw myself onto the table. I could attempt to even cut myself in the face or the throat, and before I did that, all these things would stop me. You would find a way to stop me. It’s impossible for me to die.”

[Discussed at the 5:16 mark in the associated video interview.]

Smith didn’t test his theory, but he makes a good point. The way we respond to the world often doesn’t correspond with the world’s true state. And he’s right about that not-letting-him-die thing; myself and the other people in the room would have jumped in had he crashed through a pane of glass. He would have then gone to an emergency room where the doctors and nurses would usher him through a life-saving process. The whole thing is set up to keep him among the living.

Acknowledging the safety of an environment isn’t something most people do by default. Perhaps we don’t want to tempt fate. Or maybe we’re wired to identify threats even when they’re not present. This disconnect between our ancient physical responses and our modern environments is one of the things Smith explores in his book The Flinch.

Read more…

Comments: 2
Top Stories: July 9-13, 2012

Top Stories: July 9-13, 2012

Heavy data, open source strategies for businesses, and collaborating on code.

This week on O’Reilly: Jim Stogdill said data is getting heavier relative to the networks that carry it around the data center; Simon Phipps revealed open source community strategies relevant to the enterprise; and Team Geek authors Brian Fitzpatrick and Ben Collins-Sussman discussed the importance of developer collaboration.

Comment
From smartphones and continuous data comes the social MRI

From smartphones and continuous data comes the social MRI

Dr. Nadav Aharony used phone sensors to explore personal behaviors and community trends.

It’s clear at this point that the smartphone revolution has very little to do with the phone function in these devices. Rather, it’s the unique mix of sensors, always-on connectivity and mass consumer adoption that’s shaping business and culture.

Dr. Nadav Aharony (@nadavaha) tapped into this mix when he was working on a “social MRI” study in MIT’s Media Lab. Aharony, who recently joined us as part of our ongoing foo interview series, described his vision of the social MRI:

“If you think about it, the three things you take with you when you go out of your home are your keys, your wallet and your phone, so our phones are always with us. In aggregate, we can use the phones in many people’s pockets as a virtual imaging chamber. So, one aspect of the social MRI is this virtual imaging chamber that is collecting tens or hundreds of signals at the same time from members of the community.” [Discussed at 1:16]

Aharony’s work focused on 150 participants (about 75 families) that were given phones for 15 months. During that time, more than one million hours of “continuous sensing data” was gathered with the participants’ consent. The data was acquired and scrubbed under MIT’s ethics guidelines, and for extra measure, Aharony included his own data in the dataset.

Collecting the data was just the beginning. Parsing that information and creating experiments based on emerging signals is where the applications of a social MRI became significant.
Read more…

Comment: 1
A lever is always better than a lone coder

A lever is always better than a lone coder

Team Geek authors Brian Fitzpatrick and Ben Collins-Sussman on coding myths and collaboration.

If we accept that software development is a team activity (it is), the importance of collaboration and communication becomes clear. Team Geek authors Brian Fitzpatrick and Ben Collins-Sussman discuss the nuances of modern programming in this interview.

Comments: 8