An alternate perspective on data-driven decision making

The O'Reilly Radar Podcast: Tricia Wang on "thick data," purpose-driven problem solving, and building the ideal team.

In this week’s Radar Podcast episode, O’Reilly’s Roger Magoulas chatted with Tricia Wang, a global tech ethnographer and co-founder of PL Data, about how qualitative and quantitative data need to work together, reframing “data-driven decision making,” and building the ideal team.

Subscribe to the O’Reilly Radar Podcast

TuneIn, iTunes, SoundCloud, RSS

Purpose-driven problem solving

Wang stressed that quantitative and qualitative need to work together. Rather than focusing on data-driven decision making, we need to focus on the best way to identify and solve the problem at hand: the data alone won’t provide the answers:

“It’s been kind of a detriment to our field that there’s this phrase ‘data-driven decision making.’ I think oftentimes people expect that the data’s going to give you answers. Data does not give you answers; it gives you inputs. You still have to figure out how to do the translation work and figure out what the data is trying to explain, right? I think data-driven decision making does not accurately describe what data can do. Really what we should be talking about is purpose-driven problem solving with data. Read more…

Comment: 1

Signals from Strata + Hadoop World in San Jose, CA, 2015

From data-driven government to our age of intelligence, here are key insights from Strata + Hadoop World in San Jose, CA, 2015.

Experts from across the big data world came together for Strata + Hadoop World in San Jose, CA, 2015. We’ve gathered insights from the event below.

U.S. chief data scientist

With a special recorded introduction from President Barack Obama, DJ Patil talks about his new role as the U.S. government’s first ever chief data scientist, the nature of the U.S.’s emerging data-driven government, and defines his mission in leading the data-driven initiative:

“Responsibly unleash the power of data for the benefit of the American public and maximize the nation’s return on its investment in data.”


Read more…

Comment: 1

An Internet of Things that do what they’re told

Our things are getting wired together, and you're not secure if you can't control the destiny of your private information.

Barbed_wire_Richard_Leonard_Flickr

Editor’s note: The Electronic Frontier Foundation’s Cory Doctorow will be speaking at the Solid Conference in San Francisco June 23-25, 2015. Registration is now open — for more information on the program, visit the Solid website.

The digital world has been colonized by a dangerous idea: that we can and should solve problems by preventing computer owners from deciding how their computers should behave. I’m not talking about a computer that’s designed to say, “Are you sure?” when you do something unexpected — not even one that asks, “Are you really, really sure?” when you click “OK.” I’m talking about a computer designed to say, “I CAN’T LET YOU DO THAT DAVE” when you tell it to give you root, to let you modify the OS or the filesystem.

Case in point: the cell-phone “kill switch” laws in California and Minneapolis, which require manufacturers to design phones so that carriers or manufacturers can push an over-the-air update that bricks the phone without any user intervention, designed to deter cell-phone thieves. Early data suggests that the law is effective in preventing this kind of crime, but at a high and largely needless (and ill-considered) price.

To understand this price, we need to talk about what “security” is, from the perspective of a mobile device user: it’s a whole basket of risks, including the physical threat of violence from muggers; the financial cost of replacing a lost device; the opportunity cost of setting up a new device; and the threats to your privacy, finances, employment, and physical safety from having your data compromised. Read more…

Comments: 3

Postmodern security

The real challenge going forward: we can't trust anything.

A few weeks ago, I wrote about postmodern computing, and characterized it as the computing in a world of distrust.

This morning, I read Steve Bellovin’s blog post, What Must We Trust? — Bellovin explains that “modern” (my word) security is founded on the idea of a “Trusted Computing Base” (TCB), defined (in part) in the United States’ Defense Department’s Orange Book. There were parts of a system that you had to trust, and you had to guard their integrity vigilantly: the kernel, certainly, but also specific configuration files, executables, and so on.

The TCB has always been problematic, particularly since (at least initially) it did not consider the problem of network connections. But networking aside, Bellovin argues that recent events have blown the idea of a “trusted” system to bits. We’ve seen attacks against (Bellovin’s list) batteries, webcams, USB, and more. If Andromedans (Bellovin doesn’t want to say NSA) have managed to infiltrate our disk drives, what can trust mean? And it would be naive to think that this stops with devices that have disk drives. Our devices, from Fitbits to data centers, have been pwnd even before they’re built. Read more…

Comments: 3

Processing frameworks for Hadoop

How to decide which framework is best for your particular use case.

Editor’s note: Mark Grover will be part of the team teaching the tutorial Architectural Considerations for Hadoop Applications at Strata + Hadoop World in San Jose. Visit the Strata + Hadoop World website for more information on the program.

Hadoop has become the de-facto platform for storing and processing large amounts of data and has found widespread applications. In the Hadoop ecosystem, you can store your data in one of the storage managers (for example, HDFS, HBase, Solr, etc.) and then use a processing framework to process the stored data. Hadoop first shipped with only one processing framework: MapReduce. Today, there are many other open source tools in the Hadoop ecosystem that can be used to process data in Hadoop; a few common tools include the following Apache projects: Hive, Pig, Spark, Cascading, Crunch, Tez, and Drill, along with Impala and Presto. Some of these frameworks are built on top of each other. For example, you can write queries in Hive that can run on MapReduce or Tez. Another example currently under development is the ability to run Hive queries on Spark.

Amidst all of these options, two key questions arise for Hadoop users:

  1. Which processing frameworks are most commonly used?
  2. How do I choose which framework(s) to use for my specific use case?

This post will you help answer both of these questions, giving you enough context to make an educated decision regarding the best processing framework for your specific use case. Read more…

Comments: 3

How data and connectivity are changing the nature of play

Now that technology has made its way into the playroom, there are a lot of important questions we should be asking.

Building_Blocks_Holger_Zscheyge_Flickr

Playing is how we learn. Through play, we develop large and fine motor skills, refine language and social interaction, and discover important facts about everything from the cycle of life and death to the laws of physics. When we play, we test the world around us, and share and grow.

But play is changing because it’s now filled with technology. In the coming months, I’m going to be looking at how data and connectivity are changing toys and the very nature of play. I’ll be talking to designers, inventors, technologists, and educators, and publishing the results in a report for O’Reilly Media.

Here’s my thinking so far:

Until very recent times, play was a purely tangible, real-world experience. Almost every adult alive has built a tower of blocks, climbed something, chased another person, used a skipping rope, and put together a puzzle. We all know the rules of tag and hide and seek. Some of our fondest memories include creative and imaginative play. We were pirates. We were princesses. We were explorers in a new, exciting land, limited only by our own imaginations and the loving cry of parents calling us home for dinner. Read more…

Comments: 2