- How Bad Software Leads to Bad Science — 21% of scientists who write software have never received training in software development.
- Roaring Bitmaps — compressed bitmaps which tend to outperform conventional compressed bitmaps such as WAH, EWAH or Concise. In some instances, they can be hundreds of times faster and they often offer significantly better compression.
- Two Eras of the Internet: From Pull to Push (Chris Dixon) — in which the consumer becomes the infinite sink for an unending and constant stream of updates, media, and social mobile local offers to swipe right on brands near you.
- Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods (PDF) — research on how well people decode visual cues. In order: Position along a common scale e.g. scatter plot; Position on identical but nonaligned scales e.g. multiple scatter plots; Length e.g. bar chart; Angle & Slope (tie) e.g. pie chart; Area e.g. bubbles; Volume, density, and color saturation (tie) e.g. heatmap; Color hue e.g. newsmap. (via Flowing Data)
In this O'Reilly Radar Podcast: David Rose on fairy tale inspiration, and Simon King on designing for future context.
In this podcast episode, David Rose, an instructor at MIT’s Media Lab and CEO at Ditto Labs, sits down with Mary Treseler, O’Reilly’s director of strategic content for our design space. In the interview, Rose defines his mission: “to make technology more elegant, more embedded, and hopefully, more humane.” Technology itself isn’t what drives Rose — he’s looking for inspiration in places that have captured and fueled our imaginations for centuries:
“I’m trying to be very, sort of, fairy-tale driven rather than tech driven. In the book [Enchanted Objects], I go back to some of the patterns that are revealed through Hans Christian Andersen or the Brothers Grimm or other pop culture, like spy culture or Harry Potter or Frodo, and I try to think about what those technologies are or how those services are transferable from one person to another.
“Super powers like Superman’s ability to fly don’t count because he can’t give that to anyone else, but if it’s boots that allow you to walk many miles that you wouldn’t otherwise be able to walk or a purse that replenishes or a magic carpet that could transport anybody, those qualify because those are objects that can be used by many people. I have gone back, studied these crystal balls and other objects of enchantment and magic, and think about how those could be used as a way to inspire the inventors of The Internet of Things today.”
Behind the scenes, there's a lot more to bitcoin and blockchain than first meets the eye.
Editor’s note: this is an excerpt from Chapter 1 of our recently released book Mastering Bitcoin, by Andreas Antonopoulos. You can read the full chapter here. Antonopoulos will be speaking at our upcoming event Bitcoin & the Blockchain, January 27, 2015, in San Francisco. Find out more about the event and reserve your spot here.Bitcoin is a collection of concepts and technologies that form the basis of a digital money ecosystem. Units of currency called bitcoins are used to store and transmit value among participants in the bitcoin network. Bitcoin users communicate with each other using the bitcoin protocol, primarily via the Internet; although, other transport networks can also be used. The bitcoin protocol stack, available as open source software, can be run on a wide range of computing devices, including laptops and smartphones, making the technology easily accessible.
Users can transfer bitcoin over the network to do just about anything that can be done with conventional currencies, such as buy and sell goods, send money to people or organizations, or extend credit. Bitcoin technology includes features that are based on encryption and digital signatures to ensure the security of the bitcoin network. Bitcoins can be purchased, sold, and exchanged for other currencies at specialized currency exchanges. Bitcoin, in a sense, is the perfect form of money for the Internet because it is fast, secure, and borderless. Read more…
A look at a few ways humans mesh with the rest of our data systems.
Here’s a look at a few of the ways that humans — still the ultimate data processors — mesh with the rest of our data systems: how computational power can best produce true cognitive augmentation.
Deciding betterOver the past decade, we fitted roughly a quarter of our species with sensors. We instrumented our businesses, from the smallest market to the biggest factory. We began to consume that data, slowly at first. Then, as we were able to connect data sets to one another, the applications snowballed. Now that both the front office and the back office are plugged into everything, business cares. A lot.
While early adopters focused on sales, marketing, and online activity, today, data gathering and analysis is ubiquitous. Governments, activists, mining giants, local businesses, transportation, and virtually every other industry lives by data. If an organization isn’t harnessing the data exhaust it produces, it’ll soon be eclipsed by more analytical, introspective competitors that learn and adapt faster.
Whether we’re talking about a single human made more productive by a smartphone-turned-prosthetic-brain, or a global organization gaining the ability to make more informed decisions more quickly, ultimately, Strata + Hadoop World has become about deciding better.
What does it take to make better decisions? How will we balance machine optimization with human inspiration, sometimes making the best of the current game and other times changing the rules? Will machines that make recommendations about the future based on the past reduce risk, raise barriers to innovation, or make us vulnerable to improbable Black Swans because they mistakenly conclude that tomorrow is like yesterday, only more so? Read more…
Joi Ito on the evolution of manufacturing.
Editor’s note: this interview with Joichi Ito is an excerpt from our recent report, When Hardware Meets Software, by Mike Barlow. The report looks into the new hardware movement, telling its story through the people who are building it. For more stories on the evolving relationship between software and hardware, download the free report.Joichi Ito is the director of the MIT Media Lab. Ito, who is also co-chair of the O’Reilly Solid Conference, recalls sending a group of MIT students to Shenzhen so they could see for themselves how manufacturing is evolving. “Once they got their heads around the processes in a deep way, they understood the huge differences between prototyping and manufacturing. Design for prototyping and design for manufacturing are fundamentally different,” says Ito. The problem in today’s world, according to Ito, is that “we have abstracted industrial design to the point where we think that we can just throw designs over a wall” and somehow they will magically reappear as finished products.
The trip to Shenzhen helped the students understand the manufacturing process from start to finish. “In Shenzhen, they have a $12 phone. It’s amazing. It has no screws holding it together. It’s clearly designed to be as cheap as possible. It’s also clearly designed by someone who really understands manufacturing and understands what consumers want.”
Ito also sees a significant difference between what’s happening on the factory floors in Shenzhen and the maker movement. “We’re not talking about low-volume, DIY manufacturing,” he says. Instead, Ito’s students are working through the problems and challenges of a real, live paradigm shift — the kind of gut-wrenching upheaval described in Thomas S. Kuhn’s seminal book, The Structure of Scientific Revolutions. From Kuhn’s point of view, a paradigm shift isn’t a cause for celebration or blithe headlines — it’s a sharp and unexpected blow that topples old theories, wrecks careers, and sweeps aside entire fields of knowledge. Read more…
In this O'Reilly Data Show Podcast: Sarah Meiklejohn on analytic applications for blockchain and cryptocurrency technology.
Editor’s note: we’ll explore present and future applications of cryptocurrency and blockchain technologies at our upcoming Radar Summit: Bitcoin & the Blockchain on Jan. 27, 2015, in San Francisco.
A few data scientists are starting to play around with cryptocurrency data, and as bitcoin and related technologies start gaining traction, I expect more to wade in. As the space matures, there will be many interesting applications based on analytics over the transaction data produced by these technologies. The blockchain — the distributed ledger that contains all bitcoin transactions — is publicly available, and the underlying data set is of modest size. Data scientists can work with this data once it’s loaded into familiar data structures, but producing insights requires some domain knowledge and expertise.
I recently spoke with Sarah Meiklejohn, a lecturer at UCL, and an expert on computer security and cryptocurrencies. She was part of an academic research team that studied pseudo-anonymity (“pseudonymity”) in bitcoin. In particular, they used transaction data to compare “potential” anonymity to the “actual” anonymity achieved by users. A bitcoin user can use many different public keys, but careful research led to a few heuristics that allowed them to cluster addresses belonging to the same user:
“In theory, a user can go by many different pseudonyms. If that user is careful and keeps the activity of those different pseudonyms separate, completely distinct from one another, then they can really maintain a level of, maybe not anonymity, but again, cryptographically it’s called pseudo-anonymity. So, if they are a legitimate businessman on the one hand, they can use a certain set of pseudonyms for that activity, and then if they are dealing drugs on Silk Road, they might use a completely different set of pseudonyms for that, and you wouldn’t be able to tell that that’s the same user.