But even as I’ve been cheering, I’ve had this nagging doubt in the back of my head. So I can 3D-print my own chess set. Cool. So what? Sure, you can do great things with them (enclosures for projects; every DIY-bio lab I’ve visited has a 3D printer stashed somewhere). While a 3D printer is an important step in bringing 21st-century tooling to the home hacker, they’re still fairly limited.

Last night, the other shoe dropped. Otherfab, a project of Saul Griffiths’ Otherlab, has a new Kickstarter project for Othermill: a home computer-controlled milling machine. A milling machine is a large, versatile beast that uses a high-speed cutting bit to sculpt material (often metal) into the desired shape. Instead of adding layers of plastic or some other material, like a 3D printer, a milling machine cuts material away. If you’ve ever visited machine shops, you know that milling machines are where the magic happens. Particularly state-of-the-art computer controlled mills. They’re big, they’re expensive, and they can do just about anything. Putting one in the home shop — that’s revolutionary. A printer combined with a mill (additive and subtractive processes): that’s an exciting combination.

Otherfab’s mill is intended for making custom printed circuit boards; in a home environment, cutting away unneeded copper is much preferable to using acids to etch boards (I’ve made my own boards, and I know), and gives you more immediate feedback than sending a design off to a fabrication facility. But I don’t really think this is about PC boards and electronics. As the Kickstarter points out, their mill can be used to make anything that fits: it cuts metal, wood, wax, and plastic. I can’t wait to see what people use it for. And if we’re going to get serious about reinventing and re-envisioning manufacturing, home milling machines are essential infrastructure.

Othermill reached funding in under 24 hours; they have stretch goals ranging from $100K (already passed) up to a million. It looks like, when they have a commercially available unit, the price will be somewhere around $1500, though I’m just guessing; I’d also guess that the price will continue to drop as it did with 3D printers.

I don’t know, writing about Kickstarters could end up being too much fun.

That announcement pushed me to write another post about data skepticism. The past few days, I’ve seen a resurgence of the slogan that correlation is as good as causation, if you have enough data. And I’m worried. (And I’m not vain enough to think it’s a response to my first post about skepticism; it’s more likely an effect of Cukier’s book.) There’s a fundamental difference between correlation and causation. Correlation is a two-headed arrow: you can’t tell in which direction it flows. Causation is a single-headed arrow: A causes B, not vice versa, at least in a universe that’s subject to entropy.

Let’s do some thought experiments–unfortunately, totally devoid of data. But I don’t think we need data to get to the core of the problem. Think of the classic false correlation (when teaching logic, also used as an example of a false syllogism): there’s a strong correlation between people who eat pickles and people who die. Well, yeah. We laugh. But let’s take this a step further: correlation is a double headed arrow. So not only does this poor logic imply that we can reduce the death rate by preventing people from eating pickles, it also implies that we can harm the chemical companies that produce vinegar by preventing people from dying. And here we see what’s really happening: to remove one head of the double-headed arrow, we use “common sense” to choose between two stories: one that’s merely silly, and another that’s so ludicrous we never even think about it. Seems to work here (for a very limited value of “work”); but if I’ve learned one thing, it’s that good old common sense is frequently neither common nor sensible. For more realistic correlations, it certainly seems ironic that we’re doing all this data analysis just to end up relying on common sense.

Now let’s look at something equally hypothetical that isn’t silly. A drug is correlated with reduced risk of death due to heart failure. Good thing, right? Yes–but why? What if the drug has nothing to do with heart failure, but is really an anti-depressant that makes you feel better about yourself so you exercise more? If you’re in the “correlation is as good as causation” club, doesn’t make a difference: you win either way. Except that, if the key is really exercise, there might be much better ways to achieve the same result. Certainly much cheaper, since the drug industry will no doubt price the pills at $100 each. (Tangent: I once saw a truck drive up to an orthopedist’s office and deliver Vioxx samples with a street value probably in the millions…) It’s possible, given some really interesting work being done on the placebo effect, that a properly administered sugar pill will make the patient feel better and exercise, yielding the same result. (Though it’s possible that sugar pills only work as placebos if they’re expensive.) I think we’d like to know, rather than just saying that correlation is just as good as causation, if you have a lot of data.

Perhaps I haven’t gone far enough: with enough data, and enough dimensions to the data, it would be possible to detect the correlations between the drug, psychological state, exercise, and heart disease. But that’s not the point. First, if correlation really is as good as causation, why bother? Second, to analyze data, you have to collect it. And before you collect it, you have to decide what to collect. Data is socially constructed (I promise, this will be the subject of another post), and the data you don’t decide to collect doesn’t exist. Decisions about what data to collect are almost always driven by the stories we want to tell. You can have petabytes of data, but if it isn’t the right data, if it’s data that’s been biased by preconceived notions of what’s important, you’re going to be misled. Indeed, any researcher knows that huge data sets tend to create spurious correlations.

Causation has its own problems, not the least of which is that it’s impossible to prove. Unfortunately, that’s the way the world works. But thinking about cause and how events relate to each other helps us to be more critical about the correlations we discover. As humans we’re storytellers, and an important part of data work is building a story around the data. Mere correlations arising from a gigantic pool of data aren’t enough to satisfy us. But there are good stories and bad ones, and just as it’s possible to be careful in designing your experiments, it’s possible to be careful and ethical in the stories you tell with your data. Those stories may be the closest we get ever get to an understanding of cause; but we have to realize that they’re just stories, that they’re provisional, and that better evidence (which may just be correlations) may force us to retell our stories at any moment. Correlation is as good as causation is just an excuse for intellectual sloppiness; it’s an excuse to replace thought with an odd kind of “common sense,” and to shut down the discussion that leads to good stories and understanding.

Our starting point was remote: Q told me about Tyler Brulé’s travel writing for Financial Times (behind a paywall, unfortunately), in which he says that a club sandwich is a good proxy for hotel quality: you go into the restaurant and order a club sandwich. A club sandwich isn’t hard to make: there’s no secret recipe or technique that’s going to make Hotel A’s sandwich significantly better than B’s. But it’s easy to cut corners on ingredients and preparation. And if a hotel is cutting corners on their club sandwiches, they’re probably cutting corners in other places.

This reminded me of when my daughter was in first grade, and we looked (briefly) at private schools. All the schools talked the same talk. But if you looked at classes, it was pretty clear that the quality of the music program was a proxy for the quality of the school. After all, it’s easy to shortchange music, and both hard and expensive to do it right. Oddly enough, using the music program as a proxy for evaluating school quality has continued to work through middle school and (public) high school. It’s the first thing to cut when the budget gets tight; and if a school has a good music program with excellent teachers, they’re probably not shortchanging the kids elsewhere.

How does this connect to data science? What are the proxies that allow you to evaluate a data science program from the “outside,” on the information that you might be able to cull from company blogs, a job interview, or even a job posting? We came up with a few ideas:

• Are the data scientists simply human search engines, or do they have real projects that allow them to explore and be curious? If they have management support for learning what can be learned from the organization’s data, and if management listens to what they discover, they’re accomplishing something significant. If they’re just playing Q&A with the company data, finding answers to specific questions without providing any insight, they’re not really a data science group.
• Do the data scientists live in a silo, or are they connected with the rest of the company? In Building Data Science Teams, DJ Patil wrote about the value of seating data scientists with designers, marketers, with the entire product group so that they don’t do their work in isolation, and can bring their insights to bear on all aspects of the company.
• When the data scientists do a study, is the outcome predetermined by management? Is it OK to say “we don’t have an answer” or to come up with a solution that management doesn’t like? Granted, you aren’t likely to be able to answer this question without insider information.
• What do job postings look like? Does the company have a mission and know what it’s looking for, or are they asking for someone with a huge collection of skills, hoping that they will come in useful? That’s a sign of data science cargo culting.
• Does management know what their tools are for, or have they just installed Hadoop because it’s what the management magazines tell them to do? Can managers talk intelligently to data scientists?
• What sort of documentation does the group produce for its projects? Like a club sandwich, it’s easy to shortchange documentation.
• Is the business built around the data? Or is the data science team an add-on to an existing company? A data science group can be integrated into an older company, but you have to ask a lot more questions; you have to worry a lot more about silos and management relations than you do in a company that is built around data from the start.

Coming up with these questions was an interesting thought experiment; we don’t know whether it holds water, but we suspect it does. Any ideas and opinions?

I don’t see this as a dichotomy (and, to be fair, I’m not sure that the author does either). I expect to see both, and I’d like to think a bit more about what these two apparently opposing sides mean.

Push back is inevitable. I hope there’s a significant push back, and that it has some results. Not because I’m a Glass naysayer, but because we, as technology users, are abused so often, and push back so weakly, that it’s not funny. Facebook does something outrageous; a few technorati whine; they add option 1023 to their current highly intertwined 1022 privacy options that have been designed so they can’t be understood or used effectively; and sooner or later, it all dies down. A hundred fifty users have left Facebook, and half a million more have joined. When Apple puts another brick in their walled garden, a few dozen users (myself included) bitch and moan, but does anyone leave? Personally, I’m tired of getting warnings whenever I install software that doesn’t come from the Apple Store (I’ve used the Store exactly twice), and I absolutely expect that a not-too-distant version of OS X won’t allow me to install software from “untrusted” sources, including software I’ve written. Will there be push back? Probably. Will it be effective? I don’t know; if things go as they are now, I doubt it.

There will be push back against Glass; and that’s a good thing. I think Google, of all the companies out there, is most likely to listen and respond positively. I say that partly because of efforts like the Data Liberation Front, and partly because Eric Schmidt has acknowledged that he finds many aspects of Glass creepy. But going beyond Glass: As a community of users, we need to empower ourselves to push back. We need to be able to push back effectively against Google, but more so against Apple, Facebook, and many other abusers of our data, rather than passively accept the latest intrusion as an inevitability. If Glass does nothing more than teach users that they can push back, and teach large corporations how to respond constructively, it will have accomplished much.

Is Glass the future? Yes; at least, something like Glass is part of the future. As a species, we’re not very good at putting our inventions back into the box. About three years ago, there was a big uptick in interest in augmented reality. You probably remember: Wikitude, Layar, and the rest. You installed those apps on your phone. They’re still there. You never use them (at least, I don’t). The problem with consumer-grade AR up until now has been that it was sort of awkward walking around looking at things through your phone’s screen. (Commercial AR–heads-up displays and the like–is a completely different ball game.) Glass is the first attempt at broadly useful platform for consumer AR; it’s a game changer.

Could Glass fail? Sure; I know more failed startups than I can count where the engineers did something really cool, and when they released it, the public said “what is that, and why do you think we’d want it?” Google certainly isn’t immune from that disease, which is endemic to an engineering-driven culture; just think back to Wave. I won’t deny that Google might shelve Glass if they consider unproductive, as they’ve shelved many popular applications. But I believe that Google is playing long-ball here, and thinking far beyond 2014 or 2015. In a conversation about Bitcoin last week, I said that I doubt it will be around in 20 years. But I’m certain we will have some kind of distributed digital currency, and that currency will probably look a lot like Bitcoin. Glass is the same. I have no doubt that something like Glass is part of our future. It’s a first, tentative, and very necessary step into a new generation of user interfaces, a new way of interacting with computing systems and integrating them into our world. We probably won’t wear devices around on our glasses; it may well be surgically implanted. But the future doesn’t happen if you only talk about hypothetical possibilities. Building the future requires concrete innovation, building inconvenient and “creepy” devices that nevertheless point to the next step. And it requires people pushing back against that innovation, to help developers figure out what they really need to build.

Glass will be part of our future, though probably not in its current form. And push back from users will play an essential role in defining the form it will eventually take.

Glowing plants are a curiosity; I don’t take seriously the idea that trees will be an alternative to streetlights any time in the near future. But that’s not the point. What’s exciting is that an important and serious biology project can take place in a biohacking lab, rather than in a university or an industrial facility. It’s exciting that this project could potentially become a business; I’m sure there’s a boutique market for glowing roses and living nightlights, if not for biological street lighting. And it’s exciting that we can make new things out of biological parts.

In a conversation last year, Drew Endy said that he wanted synthetic biology to “stay weird,” and that if in ten years, all we had accomplished was create bacteria that made oil from cellulose, we will have failed. Glowing plants are weird. And beautiful. Take a look at their project, fund it, and be the first on your block to have a self-illuminating garden.

But there’s another thread of data skepticism that’s profoundly important. On her MathBabe blog, Cathy O’Neil has written several articles about lying with data — about intentionally developing models that don’t work because it’s possible to make more money from a bad model than a good one. (If you remember Mel Brooks’ classic “The Producers,” it’s the same idea.) In a slightly different vein, Cathy argues that making machine learning simple for non-experts might not be in our best interests; it’s easy to start believing answers because the computer told you so, without understanding why those answers might not correspond with reality.

I had a similar conversation with David Reiley, an economist at Google, who is working on experimental design in social sciences. Heavily paraphrasing our conversation, he said that it was all too easy to think you have plenty of data, when in fact you have the wrong data, data that’s filled with biases that lead to misleading conclusions. As Reiley points out (pdf), “the population of people who sees a particular ad may be very different from the population who does not see an ad”; yet, many data-driven studies of advertising effectiveness don’t take this bias into account. The idea that there are limitations to data, even very big data, doesn’t contradict Google’s mantra that more data is better than smarter algorithms; it does mean that even when you have unlimited data, you have to be very careful about the conclusions you draw from that data. It is in conflict with the all-too-common idea that, if you have lots and lots of data, correlation is as good as causation.

Skepticism about data is normal, and it’s a good thing. If I had to give a one line definition of science, it might be something like “organized and methodical skepticism based on evidence.” So, if we really want to do data science, it has to be done by incorporating skepticism. And here’s the key: data scientists have to own that skepticism. Data scientists have to be the biggest skeptics. Data scientists have to be skeptical about models, they have to be skeptical about overfitting, and they have to be skeptical about whether we’re asking the right questions. They have to be skeptical about how data is collected, whether that data is unbiased, and whether that data — even if there’s an inconceivably large amount of it — is sufficient to give you a meaningful result.

Because the bottom line is: if we’re not skeptical about how we use and analyze data, who will be? That’s not a pretty thought.

Given the timing, the name is ironic. I’d definitely like an application similar to Evernote, but with search that actually worked well; I trust Google on search. But why should I use Keep if the chances are that Google is going to drop it a year or two from now?

In the larger scheme of things, Keep is small potatoes. Google is injuring themselves in ways that are potentially much more serious than the success or failure of one app. Google is working on the most ambitious re-envisioning of computing since the beginning of the PC era: moving absolutely everything to the cloud. Minimal local storage; local disk drives, whether solid state or rust-based, are the problem, not the solution. Projects like Google Fiber show that they’re interested in seeing that people have enough bandwidth so that they can get at their cloud storage fast enough so that they don’t notice that it isn’t local.

It’s a breath-taking vision, on many levels: I should be able to have access to all of my work, regardless of the device I’m using or where it’s located. A mobile phone shouldn’t be any different from a desktop. I may not want to write software on a mobile phone (I can’t imagine coding on those tiny touch keyboards), but I should be able to if I want to. And I should definitely be able to take a laptop into the hills and work transparently over a 4G network.

Furthermore, why should I worry about local storage? The most common cause for throwing a computer on the bone pile is disk drive failure. Granted, I keep machines around for a long time, so by the time the disk drive fails, it’s more than time for an upgrade. But local disks require backups; backups are a pain; and it’s all too common for something to go wrong when you’re doing a restore. I’d prefer to leave backups to a professional in a data center. For that matter, there are many things I’d rather leave to a data center ops group: malware detection, authentication, software updates, you name it. Most of the things that make computing a pain disappear when you move them to the cloud.

So I’ve written two paragraphs about what’s wonderful about Google’s vision. Here’s what sucks. How can I contemplate moving everything to the cloud, especially Google’s cloud, if services are going to flicker in and out of existence at the whim of Google’s management? That’s a non-starter. Google has scrapped services in the past, and though I’ve been sympathetic with the people who complained about the cancellation, they’ve been services that haven’t reached critical mass. You can’t say that about Google Reader. And if they’re willing to scrap Google Reader, why not Google Docs? I bet more people use Reader than Docs. What if they kill the Prediction API, and you rely on that? There are alternatives to Reader, there may be alternatives to Docs (though most of the ones I knew have died on the vine), but I don’t know of anything remotely like the Prediction API. I could go on with “what ifs” forever (Authentication API? Web Optimizer?), but you get the point.

If Google is serious about providing a platform that lets us move all of our computing to the cloud, they need to provide a stable platform. So far, the tools are great, but Google gets a #fail for stability. Google understands the Internet far better than its competitors, but they’re demonstrating that they don’t understand their users. If you’re a product company, taking out the trash–cancelling the old projects, the non-productive products–is an unpleasant necessity. But Google is trying to be far more than a product company. They’re trying to become a platform company, and they don’t yet understand that’s a different game, with different rules.

What makes this game unique is that you’re not playing against the other players, you’re playing against the game itself. It’s almost impossible to win, particularly at higher levels of difficulty (which Greg encourages, even for newbies). But you quickly realize that you don’t have a chance of winning if you don’t cooperate with the other players. The game is all about cooperation and collaboration. The players don’t all have equal abilities; one can move other players’ pieces around on the board, another can create research centers, another can cure larger swaths of disease. On your turn, you could just move and do whatever you think is best; but once you get the hang of it, you spend a good bit of time before each move discussing with the other players what the best strategy is, whether there are other effective ways to accomplish the same goal, and so on. You’re always discussing whether it would be better to solve a problem yourself, or move someone else so they can solve the problem more effectively on their turn.

In some ways, it’s not all that different from a role-playing game, but there is never any advantage to stabbing another player in the back or striking out on your own. But at the same time, even though it’s radically collaborative, it’s challenging. As I said, it’s almost impossible to win, and the game is structured to become more difficult the longer it goes on.

It’s a great example of rethinking gaming and rethinking competition, all in a little game that comes in a box and is played with pawns on a board.

You don’t have to read much industry news to get the sense that “big data” is sliding into the trough of Gartner’s hype curve. That’s natural. Regardless of the technology, the trough of the hype cycle is driven by by a familiar set of causes: it’s fed by over-agressive marketing, the longing for a silver bullet that doesn’t exist, and the desire to spout the newest buzzwords. All of these phenomena breed cynicism. Perhaps the most dangerous is the technologist who never understands the limitations of data, never understands what data isn’t telling you, or never understands that if you ask the wrong questions, you’ll certainly get the wrong answers.

Big data is not a term I’m particularly fond of. It’s just data, regardless of the size. But I do like Roger Magoulas’ definition of “big data”: big data is when the size of the data becomes part of the problem. I like that definition because it scales. It was meaningful in 1960, when “big data” was a couple of megabytes. It will be meaningful in 2030, when we all have petabyte laptops, or eyeglasses connected directly to Google’s yottabyte cloud. It’s not convenient for marketing, I admit; today’s “Big Data!!! With Hadoop And Other Essential Nutrients Added” is tomorrow’s “not so big data, small data actually.” Marketing, for better or for worse, will deal.

Whether or not Moore’s Law continues indefinitely, the real importance of the amazing increase in computing power over the last six decades isn’t that things have gotten faster; it’s the size of the problems we can solve has gotten much, much larger. Or as Chris Gaun just wrote, big data is leading scientists to ask bigger questions. We’ve been a little too focused on Amdahl’s law, about making computing faster, and not focused enough on the reverse: how big a problem can you solve in a given time, given finite resources? Modern astronomy, physics, and genetics are all inconceivable without really big data, and I mean big on a scale that dwarfs Amazon’s inventory database. At the edges of research, data is, and always will be, part of the problem. Perhaps even the biggest part of the problem.

In the next year, we’ll slog through the cynicism that’s a natural outcome of the hype cycle. But I’m not worrying about cynicism. Data isn’t like Java, or Rails, or any of a million other technologies; data has been with us since before computers were invented, and it will still be with us when we move onto whatever comes after digital computing. Data, and specifically “big data,” will always be at the edges of research and understanding. Whether we’re mapping the brain or figuring out how the universe works, the biggest problems will almost always be the ones for which the size of the data is part of the problem. That’s an invariant. That’s why I’m excited about data.

But the Maker movement is not just about hobbyists. We’ve seen a lot in print about the re-shoring of American manufacturing, the return of the manufacturing jobs that had been exported to China and the Far East over the past few decades. One of the questions we’re asking at O’Reilly is what the Maker movement has to do with the return of manufacturing. If the return of manufacturing just means lots of low-level industrial jobs, paying barely more than minimum wage and under near-slavery conditions, that doesn’t sound desirable. That also doesn’t sound possible, at least to me: whatever else one might say about the cost of doing business in the U.S., North America just doesn’t have the sheer concentrations of people needed to make a Foxconn.

Of course, many of the writers who’ve noted the return of manufacturing have also noted that it’s returning in a highly automated way: instead of people running around a warehouse, you’ll have Kiva robots doing the running. Instead of skilled machinists operating milling machines, you’ll have highly automated computer controlled machines with a small number of humans to test the parts and make sure they’re operating properly. This vision is more plausible — even likely — but while it promises continued employment for the engineers who make the robots, it certainly doesn’t solve any problems in the labor market.

But just as small business has long been the cornerstone of the U.S. economy, one wonders whether or not small manufacturing, driven by “professional Makers,” could be the foundation for the resurgence of manufacturing in the U.S. A number of innovations have made this shift conceivable. One of the most important is the ease with which makers can raise money to get a business started. Thanks to Kickstarter, initial funding for a small business is a lot easier than it used to be. Kickstarter isn’t alone; IndieGoGo, Selfstarter, and many others also enable Makers to raise money without running the venture capital gauntlet.

There’s also been an amazing drop in the cost of tooling. Not long ago, 3D printers, laser cutters, and computer-controlled milling machines were tools that enthusiasts could only dream about. Now you can get a 3D printer for a few hundred dollars, and a laser cutter for a couple of thousand. If you don’t want to own your own 3D printer, they’re starting to appear in storefronts and copy shops. Online fabrication services exist for everything from printed circuit boards to DNA. You design what you want online, click a button, and a few weeks later, a batch of PC boards, or 3D printed parts, or plasmids with custom DNA, arrive. This isn’t new, but it’s becoming easier all the time. Autodesk has apps for your iPad that let you design for a 3D printer; you can easily send the design to the copy shop or library for production.

In the 20th-century economy, one barrier to starting a new business was establishing a sales channel. That’s another problem that’s been solved recently. There are new outlets and sales channels that specialize in micro-manufacturing. Etsy is the most well known; Tindie is a newer entry that caters to electronics; and I believe we will see many more online marketplaces specializing in small manufacturers.

There’s more at stake in re-invigorating small manufacturing than just adding to the economy. Several years ago, I was in a meeting with Bunnie Huang, founder of Chumby, where he said that the United States had lost the engineering skills needed to do manufacturing. The engineers needed to do product development, to take a raw design and figure out how to produce it, no longer existed in the U.S., at least in sufficient numbers to support a manufacturing economy. As manufacturing had gone offshore, so had the people who knew how to do it. A product like the iPhone isn’t manufactured in China because it’s cheaper; it’s manufactured in China because that kind of manufacturing just can’t be done in the United States. Part of a reboot of American manufacturing means home-growing the product engineering and development smarts that we’ve lost over the years; and professional Making, Makers turning their ideas and passions into products, is necessary to re-develop the talent and experience that are in short supply.

If you’re a professional maker, we’d like to hear your story. What kind of a business are you running? Do you have, or foresee having, employees? What kind of an impact has your business had on your community? I’ve seen too many small towns going to ruin around an abandoned factory. The people with the skills are still there, but the jobs left years ago. Can the Maker movement make an appreciable change in local economies? And if small numbers of makers can contribute to a local economy, what can the entire movement do for the national economy?

We’re waiting for your answers.