In fact the 3Doodler rejects quite a lot of what most people would consider necessary for it to be called a 3D printer. There is no three-axis control. There is no software. You can’t download a design and print an object. It strips 3D printing back to basics.

What there is, what it allows you to do, is make things. This is the history of printing going in reverse. It’s as if Gutenberg’s press was invented first, and then somebody came along afterwards and invented the fountain pen.

While the 3Doodler looks simple, the creators have obviously overcome some serious technological difficulties to get it working. One of the things that’s hard to do on 3D printers, at least hard to do well, is unsupported structures.

As anyone that owns a 3D printer will tell you, the cooling time for the plastic as it leaves the print head is crucial to allow you to print unsupported structures. Too hot and it doesn’t work, the structure sags and runs. Too cold and it just plain doesn’t work at all. From their videos, the 3Doodler inventors seem to have cracked the problem. Building a free-standing structure appears to be easy and well within the capabilities of the pen.

It also takes 3mm ABS and PLA as its “ink,” the same stuff used by most hobbyist 3D printers. I’ve got spools of this stuff hanging around my house, which I use in my own printer. But unlike my printer, which cost just under a thousand dollars, the 3Doodler costs just $75.

It doesn’t have the same capabilities, but that’s the difference between a printing press and a pen. It has different capabilities, ones a “normal” 3D printer doesn’t have. It’s not a cheap alternative, it’s a different thing entirely.

I’m currently watching the 3Doodler climb past its first million dollars on Kickstarter. When I say its “first” million I mean that. The project has more than 30 days left on its campaign and already it’s gone viral. This is the next Pebble. The next Kickstarter success story.

The creators have tapped into a previously untappable market: People who wanted a 3D printer but couldn’t afford one, and people who see the obvious potential of a fountain pen over a printing press, for both art and engineering.

The guys behind the 3Doodler made $60,000 dollars while I wrote this post. My hat is off to them. It’s not often someone comes up with an idea this good.

I’m going to be writing a series of posts on hardware startups over the course of the next few months, and rest assured I’ll come back to the 3Doodler. But not until they can type faster than they can make money.

Well, yes, that’s arguable. But most people, everyday people, don’t know that. It doesn’t matter if the paradigm survives if it’s not culturally acknowledged. Mainframe computing lives on, buried behind the scenes, backstage. As a platform it performs well, in its own niche. No doubt desktop computing is destined to live on, but similarly behind the scenes, and it’s already fading into the background.

The desktop will increasingly belong to niche users. Developers need them, at least for now and for the foreseeable future. But despite the prevalent view in Silicon Valley, the world does not consist of developers. Designers need screen real estate, but buttons and the entire desktop paradigm are a hack; I can foresee the day when the computing designers use will not even vaguely resemble today’s desktop machines.

For the rest of the world? Computing will almost inevitably diffuse out into our environment. Today’s mobile devices are transition devices, artifacts of our stage of technology progress. They too will eventually fade into their own niche. Replacement technologies, or rather user interfaces, like Google’s Project Glass are already on the horizon, and that’s just the beginning.

People never wanted computers; they wanted what computers could do for them. Almost inevitably the amount computers can do for us on their own, behind our backs, is increasing. But to do that, they need data, and to get data they need sensors. So the diffusion of general purpose computing out into our environment is inevitable.

Everyday objects are already becoming smarter. But in 10 years’ time, every piece of clothing you own, every piece of jewelry, and every thing you carry with you will be measuring, weighing and calculating. In 10 years, the world — your world — will be full of sensors.

The sensors you carry with you may well generate more data every second, both for you and about you, than previous generations did about themselves during the course of their entire lives. We will be surrounded by a cloud of data. While the phrase “data exhaust” has already entered the lexicon, we’re still essentially at the banging-the-rocks-together stage. You haven’t seen anything yet …

The end point of this evolution is already clear: it’s called smart dust. General purpose computing, sensors, and wireless networking, all bundled up in millimeter-scale sensor motes drifting in the air currents, flecks of computing power, settling on your skin, ingested, will be monitoring you inside and out, sensing and reporting — both for you and about you.

Almost inevitably the amount of data that this sort of technology will generate will vastly exceed anything that can be filtered, and distilled, into a remote database. The phrase “data exhaust” will no longer be a figure of speech; it’ll be a literal statement. Your data will exist in a cloud, a halo of devices, tasked to provide you with sensor and computing support as you walk along, calculating constantly, consulting with each other, predicting, anticipating your needs. You’ll be surrounded by a web of distributed sensors and computing.

Makes desktop computing look sort of dull, doesn’t it?

Photo: it’s not fog… it’s smoke… by Guilherme Jófili, on Flickr

Related:

• They promised us flying cars
• The next, next big thing
• The secret is to bang the rocks together

The FBI is aware of published reports alleging that an FBI laptop was compromised and private data regarding Apple UDIDs was exposed. At this time there is no evidence indicating that an FBI laptop was compromised or that the FBI either sought or obtained this data.

Of course that statement leaves a lot of leeway. It could be the agent’s personal laptop, and the data may well have been “property” of an another agency. The wording doesn’t even explicitly rule out the possibility that this was an agency laptop, they just say that right now they don’t have any evidence to suggest that it was.

This limited data release doesn’t have much impact, but the possible release of the full dataset, which is claimed to include names, addresses, phone numbers and other identifying information, is far more worrying.

While there are some almost dismissing the issue out of hand, the real issues here are: Where did the data originate? Which devices did it come from and what kind of users does this data represent? Is this data from a cross-section of the population, or a specifically targeted demographic? Does it originate within the law enforcement community, or from an external developer? What was the purpose of the data, and why was it collected?

With conflicting stories from all sides, the only thing we can believe is the data itself. The 40-character strings in the release at least look like UDID numbers, and anecdotally at least we have a third-party confirmation that this really is valid UDID data. We therefore have to proceed at this point as if this is real data. While there is a possibility that some, most, or all of the data is falsified, that’s looking unlikely from where we’re standing standing at the moment.

With that as the backdrop, the first action I took was to check the released data for my own devices and those of family members. Of the nine iPhones, iPads and iPod Touch devices kicking around my house, none of the UDIDs are in the leaked database. Of course there isn’t anything to say that they aren’t amongst the other 11 million UDIDs that haven’t been released.

With that done, I broke down the distribution of leaked UDID numbers by device type. Interestingly, considering the number of iPhones in circulation compared to the number of iPads, the bulk of the UDIDs were self-identified as originating on an iPad.

Distribution of UDID by device type

What does that mean? Here’s one theory: If the leak originated from a developer rather than directly from Apple, and assuming that this subset of data is a good cross-section on the total population, and assuming that the leaked data originated with a single application … then the app that harvested the data is likely a Universal application (one that runs on both the iPhone and the iPad) that is mostly used on the iPad rather than on the iPhone.

The very low numbers of iPod Touch users might suggest either demographic information, or that the application is not widely used by younger users who are the target demographic for the iPod Touch, or alternatively perhaps that the application is most useful when a cellular data connection is present.

The next thing to look at, as the only field with unconstrained text, was the Device Name data. That particular field contains a lot of first names, e.g. “Aaron’s iPhone,” so roughly speaking the distribution of first letters in the this field should give a decent clue as to the geographical region of origin of the leaked list of UDIDs. This distribution is of course going to be different depending on the predominant language in the region.

Distribution of UDID by the first letter of the “Device Name” field

The immediate stand out from this distribution is the predominance of device name strings starting with the letter “i.” This can be ascribed to people who don’t have their own name prepended to the Device Name string, and have named their device “iPhone,” “iPad” or “iPod Touch.”

The obvious next step was to compare this distribution with the relative frequency of first letters in words in the English language.

Comparing the distribution of UDID by first letter of the “Device Name” field against the relative frequencies of the first letters of a word in the English language

The spike for the letter “i” dominated the data, so the next step was to do some rough and ready data cleaning.

I dropped all the Device Name strings that started with the string “iP.” That cleaned out all those devices named “iPhone,” “iPad” and “iPod Touch.” Doing that brought the number of device names starting with an “i” down from 159,925 to just 13,337. That’s a bit more reasonable.

Comparing the distribution of UDID by first letter of the “Device Name” field, ignoring all names that start with the string “iP,” against the relative frequencies of the first letters of a word in the English language

I had a slight over-abundance of “j,” although that might not be statistically significant. However, the stand out was that there was a serious under-abundance of strings starting with the letter “t,” which is interesting. Additionally, with my earlier data cleaning I also had a slight under-abundance of “i,” which suggested I may have been too enthusiastic about cleaning the data.

Looking at the relative frequency of letters in languages other than English it’s notable that amongst them Spanish has a much lower frequency of the use of “t.”

As the de facto second language of the United States, Spanish is the obvious next choice  to investigate. If the devices are predominantly Spanish in origin then this could solve the problem introduced by our data cleaning. As Marcos Villacampa noted in a tweet, in Spanish you would say “iPhone de Mark” rather than “Mark’s iPhone.”

Comparing the distribution of UDID by first letter of the “Device Name” field, ignoring all names that start with the string “iP,” against the relative frequencies of the first letters of a word in the Spanish language

However, that distribution didn’t really fit either. While “t” was much better, I now had an under-abundance of words with an ”e.” Although it should be noted that, unlike our English language relative frequencies, the data I was using for Spanish is for letters in the entire word, rather than letters that begin the word. That’s certainly going to introduce biases, perhaps fatal ones.

Not that I can really make the assumption that there is only one language present in the data, or even that one language predominates, unless that language is English.

At this stage it’s obvious that the data is, at least more or less, of the right order of magnitude. The data probably shows devices coming from a Western country. However, we’re a long way from the point where I’d come out and say something like ” … the device names were predominantly in English.” That’s not a conclusion I can make.

I’d be interested in tracking down the relative frequency of letters used in Arabic when the language is transcribed into the Roman alphabet. While I haven’t been able to find that data, I’m sure it exists somewhere. (Please drop a note in the comments if you have a lead.)

The next step for the analysis is to look at the names themselves. While I’m still in the process of mashing up something that will access U.S. census data and try and reverse geo-locate a name to a “most likely” geographical origin, such services do already exist. And I haven’t really pushed the boundaries here, or even started a serious statistical analysis of the subset of data released by Antisec.

This brings us to Pete Warden’s point that you can’t really anonymize your data. The anonymization process for large datasets such as this is simply an illusion. As Pete wrote:

Precisely because there are now so many different public datasets to cross-reference, any set of records with a non-trivial amount of information on someone’s actions has a good chance of matching identifiable public records.

While this release in itself is fairly harmless, a number of “harmless” releases taken together — or cleverly cross-referenced with other public sources such as Twitter, Google+, Facebook and other social media — might well be more damaging. And that’s ignoring the possibility that Antisec really might have names, addresses and telephone numbers to go side-by-side with these UDID records.

The question has to be asked then, where did this data originate? While 12 million records might seem a lot, compared to the number of devices sold it’s not actually that big a number. There are any number of iPhone applications with a 12-million-user installation base, and this sort of backend database could easily have been built up by an independent developer with a successful application who downloaded the device owner’s contact details before Apple started putting limitations on that.

Ignoring conspiracy theories, this dataset might be the result of a single developer. Although how it got into the FBI’s possession and the why of that, if it was ever there in the first place, is another matter entirely.

I’m going to go on hacking away at this data to see if there are any more interesting correlations, and I do wonder whether Antisec would consider a controlled release of the data to some trusted third party?

Much like the reaction to #locationgate, where some people were happy to volunteer their data, if enough users are willing to self-identify, then perhaps we can get to the bottom of where this data originated and why it was collected in the first place.

Thanks to Hilary Mason, Julie Steele, Irene Ros, Gemma Hobson and Marcos Villacampa for ideas, pointers to comparative data sources, and advice on visualisation of the data.

Update

9/6/12

In response to a post about this article on Google+, Josh Hendrix made the suggestion that I should look at word as well as letter frequency. It was a good idea, so I went ahead and wrote a quick script to do just that…

The top two words in the list are “iPad,” which occurs 445,111 times, and “iPhone,” which occurs 252,106 times. The next most frequent word is “iPod,” but that occurs only 36,367 times. This result backs up my earlier result looking at distribution by device type.

Then there are various misspellings and mis-capitalisations of “iPhone,” “iPad,” and “iPod.”

The first real word that isn’t an Apple trademark is “Administrator,” which occurs 10,910 times. Next are “David” (5,822), “John” (5,447), and “Michael” (5,034). This is followed by “Chris” (3,744), “Mike” (3,744), “Mark” (3,66) and “Paul” (3,096).

Looking down the list of real names, as opposed to partial strings and tokens, the first female name doesn’t occur until we’re 30 places down the list — it’s “Lisa” (1,732) with the next most popular female name being “Sarah” (1,499), in 38th place.

The top 100 names occurring in the UDID list.

The word “Dad” occurs 1,074 times, with “Daddy” occurring 383 times. For comparison the word “Mum” occurs just 58 times, and “Mummy” just 33. “Mom” came in with 150 occurrences, and “mommy” with 30. The number of occurrences for “mum,” “mummy,” “mom,” and “mommy” combined is 271, which is still very small compared to the combined total of 1,457 for “dad” and “daddy.”

[Updated: Greg Yardly wisely pointed out on Twitter that I was being a bit English-centric in only looking for the words "mum" and "mummy," which is why I expanded the scope to include "mom" and "mommy."]

There is a definite gender bias here, and I can think of at least a few explanations. The most likely is fairly simplistic: The application where the UDID numbers originated either appeals to, or is used more, by men.

Alternatively, women may be less likely to include their name in the name of their device, perhaps because amongst other things this name is used to advertise the device on wireless networks?

Either way I think this definitively pins it down as a list of devices originating in an Anglo-centric geographic region.

Sometimes the simplest things work better. Instead of being fancy perhaps I should have done this in the first place. However this, combined with my previous results, suggest that we’re looking at an English speaking, mostly male, demographic.

Correlating the top 20 or so names and with the list of most popular baby names (by year) all the way from the mid-’60s up until the mid-’90s (so looking at the most popular names for people between the ages of say 16 and 50) might give a further clue as to the exact demographic involved.

Both Gemma Hobson and Julie Steele directed me toward the U.S. Social Security Administration’s Popular Baby Names By Decade list. A quick and dirty analysis suggests that the UDID data is dominated by names that were most popular in the ’70s and ’80s. This maps well to my previous suggestion that the lack of iPod Touch usage might suggest that the demographic was older.

I’m going to do a year-by-year breakdown and some proper statistics later on, but we’re looking at an application that’s probably used by: English speaking males with an Anglo-American background in their 30s or 40s. It’s most used on the iPad, and although it also works on the iPhone, it’s used far less on that platform.

Thanks to Josh Hendrix, and again to Gemma Hobson and Julie Steele, for ideas and pointers to sources for this part of the analysis.

Update

9/11/12

A really nice analysis from David Schuetz uses the frequency of UDID duplicates and the names of those devices to track down the source of the leak. I really should have thought of that.

Interestingly, however, it does support my own analysis. Yesterday, a Florida publishing company named BlueToad said the UDID data was taken from their systems. BlueToad makes apps for magazine publishers, hence the predominance of of the iPad over the iPhone in my results, as those apps are more often used on the iPad.

Also they seem to mostly market into the U.S., which supports my ethnicity findings, and looking at the list of titles they curate, it does look like my demographics are more-or-less spot on as well. Those look like magazines marketed to men in their 30s and 40s to me.

I’d actually been really confused about what type of app could possibly have that narrow a demographic, and this sort of clears up my confusion. Nice!

Related:

• Got an iPhone or 3G iPad? Apple is recording your moves
• Radiation visualizations paint a different picture of Japan

NASA’s Astrophysics Data System (ADS), hosted by the Smithsonian Astrophysical Observatory (SAO), has quietly been working away since the mid-’90s. Without much, if any, fanfare amongst the other disciplines, it has moved astronomers into a world where access to the literature is just a given. It’s something they don’t have to think about all that much.

The ADS service provides access to abstracts for virtually all of the astronomical literature. But it also provides access to the full text of more than half a million papers, going right back to the start of peer-reviewed journals in the 1800s. The service has links to online data archives, along with reference and citation information for each of the papers, and it’s all searchable and downloadable.

The existence of the ADS, along with the arXiv pre-print server, has meant that most astronomers haven’t seen the inside of a brick-built library since the late 1990s.

It also makes astronomy almost uniquely well placed for interesting data mining experiments, experiments that hint at what the rest of academia could do if they followed astronomy’s lead. The fact that the discipline’s literature has been scanned, archived, indexed and catalogued, and placed behind a RESTful API makes it a treasure trove, both for hypothesis generation and sociological research.

For example, the .Astronomy series of conferences is a small workshop that brings together the best and the brightest of the technical community: researchers, developers, educators and communicators. Billed as “20% time for astronomers,” it gives these people space to think about how the new technologies affect both how research and communicating research to their peers and to the public is done.

[Disclosure: I'm a member of the advisory board to the .Astronomy conference, and I previously served as a member of the programme organising committee for the conference series.]

It should perhaps come as little surprise that one of the more interesting projects to come out of a hack day held as part of this year’s .Astronomy meeting in Heidelberg was work by Robert Simpson, Karen Masters and Sarah Kendrew that focused on data mining the astronomical literature.

The team grabbed and processed the titles and abstracts of all the papers from the Astrophysical Journal (ApJ), Astronomy & Astrophysics (A&A), and the Monthly Notices of the Royal Astronomical Society (MNRAS) since each of those journals started publication — and that’s 1827 in the case of MNRAS.

By the end of the day, they’d found some interesting results showing how various terms have trended over time. The results were similar to what’s found in Google Books’ Ngram Viewer.

After the meeting, however, Robert has taken his initial results and explored the astronomical literature and his new corpus of data on the literature. He’s explored various visualisations of the data, including word matrixes for related terms and for various astro-chemistry.

He’s also taken a look at authorship in astronomy and is starting to find some interesting trends.

You can see that single-author papers dominated for most of the 20th century. Around 1960, we see the decline begin, as two- and three-author papers begin to become a significant chunk of the whole. In 1978, author papers become more prevalent than single-author papers.

Here we see that people begin to outpace papers in the 1960s. This may reflect the fact that as we get more technical as a field, and more specialised, it takes more people to write the same number of papers, which is a sort of interesting result all by itself.

Interview with Robert Simpson: Behind the project and what lies ahead

I recently talked with Rob about the work he, Karen Masters, and Sarah Kendrew did at the meeting, and the work he’s been doing since with the newly gathered data.

What made you think about data mining the ADS?

Robert Simpson: At the .Astronomy 4 Hack Day in July, Sarah Kendrew had the idea to try to do an astronomy version of BrainSCANr, a project that generates new hypotheses in the neuroscience literature. I’ve had a go at mining ADS and arXiv before, so it seemed like a great excuse to dive back in.

Do you think there might be actual science that could be done here?

Robert Simpson: Yes, in the form of finding questions that were unexpected. With such large volumes of peer-reviewed papers being produced daily in astronomy, there is a lot being said. Most researchers can only try to keep up with it all — my daily RSS feed from arXiv is next to useless, it’s so bloated. In amongst all that text, there must be connections and relationships that are being missed by the community at large, hidden in the chatter. Maybe we can develop simple techniques to highlight potential missed links, i.e. generate new hypotheses from the mass of words and data.

Are the results coming out of the work useful for auditing academics?

Robert Simpson: Well, perhaps, but that would be tricky territory in my opinion. I’ve only just begun to explore the data around authorship in astronomy. One thing that is clear is that we can see a big trend toward collaborative work. In 2012, only 6% of papers were single-author efforts, compared with 70+% in the 1950s.

We can measure how large groups are becoming, and who is part of which groups. In that sense, we can audit research groups, and maybe individual people. The big issue is keeping track of people through variations in their names and affiliations. Identifying authors is probably a solved problem if we look at ORCID.

What about citations? Can you draw any comparisons with h-index data?

Robert Simpson: I haven’t looked at h-index stuff specifically, at least not yet, but citations are fun. I looked at the trends surrounding the term “dark matter” and saw something interesting. Mentions of dark matter rise steadily after it first appears in the late ’70s.

In the data, astronomy becomes more and more obsessed with dark matter — the term appears in 1% of all papers by the end of the ’80s and 6% today.

Looking at citations changes the picture. The community is writing papers about dark matter more and more each year, but they are getting fewer citations than they used to (the peak for this was in the late ’90s). These trends are normalised, so the only regency effect I can think of is that dark matter papers take more than 10 years to become citable. Either that or dark matter studies are currently in a trough for impact.

Can you see where work is dropped by parts of the community and picked up again?

Robert Simpson: Not yet, but I see what you mean. I need to build a better picture of the community and its components.

Can you build a social graph of astronomers out of this data? What about (academic) family trees?

Robert Simpson: Identifying unique authors is my next step, followed by creating fingerprints of individuals at a given point in time. When do people create their first-author papers, when do they have the most impact in their careers, stuff like that.

What tools did you use? In hindsight, would you do it differently?

I’m using Ruby and Perl to grab the data, MySQL to store and query it, JavaScript to display it (Google Charts and D3.js). I may still move the database part to MongoDB because it was designed to store documents. Similarly, I may switch from ADS to arXiv as the data source. Using arXiv would allow me to grab the full text in many cases, even if it does introduce a peer-review issue.

What’s next?

Robert Simpson: My aim is still to attempt real hypothesis generation. I’ve begun the process by investigating correlations between terms in the literature, but I think the power will be in being able to compare all terms with all terms and looking for the unexpected. Terms may correlate indirectly (via a third term, for example), so the entire corpus needs to be processed and optimised to make it work comprehensively.

Science between the cracks

I’m really looking forward to seeing more results coming out of Robert’s work. This sort of analysis hasn’t really been possible before. It’s showing a lot of promise both from a sociological angle, with the ability to do research into how science is done and how that has changed, but also ultimately as a hypothesis engine — something that can generate new science in and of itself. This is just a hack day experiment. Imagine what could be done if the literature were more open and this sort of analysis could be done across fields?

Right now, a lot of the most interesting science is being done in the cracks between disciplines, but the hardest part of that sort of work is often trying to understand the literature of the discipline that isn’t your own. Robert’s project offers a lot of hope that this may soon become easier.

Except, of course, all that has come true. Google just got the first license to drive their cars entirely autonomously on public highways. Apple came along with the iPhone and changed everything. Three-dimensional printers have come out of the laboratories and into the home. And in a few short years, and from a standing start, Elon Musk and SpaceX has achieved what might otherwise have been thought impossible: late last year, SpaceX launched a spacecraft and returned it to Earth safely. Then they launched another, successfully docked it with the International Space Station, and then again returned it to Earth.

The SpaceX Dragon capsule is grappled and berthed to the Earth-facing port of the International Space Station’s Harmony module at 12:02 p.m. EDT, May 25, 2012. Credit: NASA/SpaceX

Right now there is a generation of high-tech tinkerers breaking the seals on proprietary technology and prototyping new ideas, which is leading to a rapid growth in innovation. The members of this generation, who are building open hardware instead of writing open software, seem to have come out of nowhere. Except, of course, they haven’t. Promised a future they couldn’t have, they’ve started to build it. The only difference between them and Elon Musk, Jeff Bezos, Sergey Brin, Larry Page and Steve Jobs is that those guys got to build bigger toys than the rest of us.

The dotcom billionaires are regular geeks just like us. They might be the best of us, or sometimes just the luckiest, but they grew up with the same dreams, and they’ve finally given up waiting for governments to build the future they were promised when they were kids. They’re going to build it for themselves.

The thing that’s driving the Maker movement is the same thing that’s driving bigger shifts, like the next space race. Unlike the old space race, pushed by national pride and the hope that we could run fast enough in place so that we didn’t have to start a nuclear war, this new space race is being driven by personal pride, ambition and childhood dreams.

But there are some who don’t see what’s happening, and they’re about to miss out. Case in point: a lot of big businesses are confused by the open hardware movement. They don’t understand it, don’t think it’s worth their while to make exceptions and cater to it. Even the so-called “smart money” doesn’t seem to get it. I’ve heard moderately successful venture capitalists from the Valley say that they “… don’t do hardware.” Those guys are about to lose their shirts.

Makers are geeks like you and me who have decided to go ahead and build the future themselves because the big corporations and the major governments have so singularly failed to do it for us. Is it any surprise that dotcom billionaires are doing the same? Is it any surprise that the future we build is going to look a lot like the future we were promised and not so much like the future we were heading toward?

Related:

• The next, next big thing
• The secret is to bang the rocks together
• Utopia on a budget: A completely practical plan for regaining paradise

Sections

• The data you carry with you
• Data leakage
• Location, location, location
• Data exhaust
• Keep your friends close, and
  your enemies closer
• Data sharing
• Building platforms
• A thought to ponder

Big data isn’t just about multi-terabyte datasets hidden inside eventually-concurrent distributed databases in the cloud, or enterprise-scale data warehousing, or even the emerging market in data. It’s also about the hidden data you carry with you all the time; the slowly growing datasets on your movements, contacts and social interactions.

Until recently, most people’s understanding of what can actually be done with the data collected about us by our own cell phones was theoretical. There were few real-world examples. But over the last couple of years, this has changed dramatically. Courting hubris perhaps, but I must admit it’s possible some of that was my fault, though I haven’t been alone.

The data you carry with you

You probably think you know how much data you carry around with you on your cell phone. You’ll certainly be aware of it if you’ve ever lost your phone, or had it stolen, or it’s just plain stopped working. But there is a large amount of data in the background that isn’t surfaced in the user interface.

We know about what I generally call primary data: calendars, address books, photographs, SMS messages and browser bookmarks. These are usually user generated, and we’d be pretty unhappy if we lost them. There is also the secondary data that the phone generates about us: call history, voice mail, usage information and records of our current and past locations. Most of what I’d call secondary data is surfaced to us in our phone’s user interface. We generally can’t change this sort of information without resetting the phone to a factory fresh condition; it’s generated by the device for us, it’s not something we generate ourselves.

But there is also what I refer to as tertiary data. This is data that, similar to the examples I mentioned above, is generated about us, rather than by us. Mostly, this data consists of cache files — data that is entirely necessary to you using the device, or significantly improves your user experience, but you don’t necessarily know is there. At least until some hole is found in the operating system to expose that data layer to you. That’s happened before, after all.

An obvious example is tucked in your photographs. Every picture you take is geotagged and date stamped, and if you publish your pictures to a photo-sharing site without stripping that information, you’re leaking data. Back in 2007, when geotagged photographs of newly arrived helicopters at a U.S. Army base in Iraq were published to the Internet, they allowed insurgents to determine the exact location of the helicopters inside the compound and conduct a mortar attack. Four of the AH-64 Apaches on the flight line were destroyed in the attack.

Where Conference 2012 — O’Reilly’s Where Conference, being held April 2-4 in San Francisco, is where the people working on and using location technologies explore emerging trends in software development, tools, business strategies and marketing.

Save 20% on registration with the code RADAR20

Data leakage

Recently, there have been a number of high-profile cases of data leakage, and the one that has raised the most controversy, at least until the next time, is the social network Path.

Upon opening the Path application on your phone, it automatically uploaded your address book to Path’s servers so it could find “friends” that you might want to connect to without asking for explicit permission to do so, or even implicit permission for that matter. Path has since apologized and updated its application so that it now asks permission before pushing your address book to its servers.

This was not data theft, but data leakage. You asked the application to accomplish something and didn’t really ask yourself how it was doing it. While there are technical solutions that don’t involve uploading your address book, the laziest solution is probably what you should have expected. I can almost hear the developers, “… we’ll just upload the address book for now and switch to hashing later on when we have time.”

There has been a lot of comment that somehow the whole Path thing was unexpected. Realistically, that’s not the case. It’s not an isolated circumstance, either. To the best of my knowledge Hipster and other apps also tapped your address book behind the scenes without asking permission. Interestingly, there are other, less obvious, culprits. Applications that make use of Chillingo’s “Crystal” game service, like Angry Birds, will in some circumstances also upload your address book. While there is a button to push, it is, at least for me, misleadingly labelled and doesn’t suggest what’s going to happen next.

Data leakage like this is not really a solvable problem at the user level, at least not in real-time. Having multiple permission boxes pop up at regular intervals is a bad design choice; users stop reading them, they lose importance and become ineffective. Just try using Microsoft Windows and you’ll understand exactly what I mean. Modal interrupts should be reserved for vital time-critical issues. They’re already used far to prolifically in iOS. Run the Mail application with multiple mail accounts configured when you’re not connected to the network and that’ll become instantly obvious. You’ll be bombarded by error messages.

I did have a thought that you might be able to deploy a customized web proxy directly onto your mobile device and have all web requests directed through it. The proxy would sift through the outgoing network connections in a (semi-)Bayesian manner looking for data that you don’t want transmitted and stop the application cold before it sends it to the remote server. Basically, it’s acting as a reverse spam filter, or a smart firewall, depending on how you want to think about it.

I think that something like this could well be far more effective at stopping data leakage than the current solution, which Google has used on Android: Permissions pages when you initially install an application are all very well, but most people don’t read them, and when you’re installing an application you’re not really thinking about why it might need certain permissions. However, you can be very clear about what data you’re interested in not leaving your phone. One configuration page for the proxy, rather than multiple ones, every time you install an application. Like modal dialogs on the iPhone, you subconsciously start to ignore them, to your peril.

Location, location, location

Of course, I can’t really talk about data leakage without mentioning the kerfuffle surrounding location and data privacy that happened just about this time last year. Unsurprisingly, the file in question still exists, despite some of the press stories; the existence of the file was never the problem. A cache of that nature is fairly necessary if you want to have reliable and timely location services on your phone. However, the file is now actually just that, a cache, and it is regularly swept clean by the operating system. It’s also not included in your usually unencrypted backups to your laptop, which was perhaps more of a problem than the fact it wasn’t being cleared out in the first place.

A visualization of iPhone location data. Click to enlarge.

What Apple was doing was taking a piece of tertiary data, generated about you by the device, and then exposing it on a platform (laptop or desktop) where accessing that data was easier. There are a lot of people who know how to navigate a file system on a computer, but a lot fewer who would know how to get the same data directly from the phone itself. It was a classic case of data leakage: data moved from a secure(ish) environment on the phone to a less secure one on the computer.

Data exhaust

Back in the days of floppy disks, the lines of ownership were pretty clear. If you had the disk, the data was yours. If someone else had it, it was theirs. Things these days are much blurrier. That tertiary data — data that’s generated about us but not by us — doesn’t just build up on your mobile devices of course. Other people are building datasets about our patterns of movement, buying decisions, credit worthiness and other things. The ability to compile these sorts of datasets left the realm of major governments with the invention of the computer.

We’re all aware of this, and there’s even a provocative buzzword to describe it: data exhaust. It’s the data we leave behind us, rather than carry with us.

In the U.S., data from grocery store loyalty schemes has been used by security services to search for terrorist suspects. Turns out the number of toilet rolls you buy can be quite telling.

Which does make me think, instead of being afraid of the data exhaust, perhaps we should embrace it. In the U.K., the biggest retailer is the supermarket Tesco. Like many, I spend a good fraction of my income there, and like almost everyone I know, I have a Tesco Clubcard. This is a loyalty card that has a record of (almost) every purchase I make, from toilet rolls to roast chicken.

I’d actually pay good money for a copy of my own Clubcard data, so long as it was actually in a machine-readable format, not on paper. Although for Tesco, the data is only really interesting in aggregate; it’s the fact that they have millions of Clubcard records that makes the dataset useful to the company. To me, a history of my purchases would be useful data.

Of course, people have already started selling our data exhaust back to us. Think about your credit report, for instance.

Keep your friends close, and your enemies closer

It’s not just your own data exhaust that you have to worry about. There was an interesting paper recently by Adam Sadilek of the Department of Computer Science at the University of Rochester. It talked about how geotagged tweets could be used to locate individuals, even if they themselves didn’t geotag their tweets — it was enough that their friends did so.

Geotagged messages on Twitter during a typical weekday afternoon in New York City.

The paper found that only a couple of weeks’ worth of location data on an individual, combined with location data from their two most-sharing friends, was enough to place that person within a 100-meter radius with 77% accuracy. That rises to nearly 85% when you combine information from nine friends.

Even someone who has never shared their location at all can be pinpointed with 47% accuracy from information available from two friends. That goes up to 57% when you include nine friends.

Data sharing

There is a great debate going on right now, which is really only starting to surface into the mainstream press, about how we share data. Despite social networks becoming mainstream, the recent privacy debacles in the mobile space say a lot about how users perceive information privacy. I think Sadilek’s paper presents even more compelling evidence.

For instance, I’m finding Google’s new Instant Upload feature, where photos taken on my phone are automatically uploaded to Google+ behind the scenes, a lot spookier and more worrying than I thought I would. It’s especially interesting that I’m feeling that way, as I’m using Apple’s Photo Stream without thinking or worrying about it that much.

I’m trying to figure out whether it’s because the privacy trade-off — in Apple’s case sharing my photos between all my devices, and in Google’s case making my photos more-or-less instantly available for sharing in Google+ — is more obviously in my favor with Photo Stream, or it’s for other reasons.

The interesting thing here is that Photo Stream and Instant Upload are, at least behind the scenes, effectively identical. Both are cloud services and your photos are stored in a data center somewhere. The master copies of your photos have essentially been moved to the cloud, rather than residing on your device.

However, because of the context these two services operate in, I have no problems with one, and I’m finding the other an uncomfortable fit. I think there’s a big lesson there for people dealing with personal information. When you’re sharing someone’s information, even with their informed consent, the context is important about how they think about the implications surrounding that sharing.

Building platforms

So, all of this got me thinking. There are large personal datasets about me, and you, and everyone, being built up by large companies. But we’re also building up datasets about ourselves, in our own control. What happens if we mash them together? Can we actually do something productive?

I’m currently running an interesting experiment with my credit card and my iPhone. I’m scraping my bank’s website to grab transaction data in near real-time onto one of my servers. Each transaction comes with a postcode. This is like a U.S. zip code, but it normally specifies a much smaller neighborhood, perhaps down to a single street or smaller in a major urban area.

On my iPhone, I’m running an application that continually monitors my location using the Significant Location Change service, so my phone knows my location to better than 1km (perhaps much better in a crowded city) more or less all of the time.

Every time a new transaction occurs, I forward it via push notification from the back-end server to my iPhone. Now, my iPhone knows both the location where the transaction took place and where I actually was at the time. If those locations don’t match, then this indicates there might have been a fraudulent transaction and it flags it for me with a notification.

The interesting thing here is that I’m using data that my credit card company doesn’t have, and hopefully will never have: my actual physical location when the transaction took place. They couldn’t possibly provide this service to me because they simply don’t have the data I have.

Of course, there are false positives. Online transactions in particular stand out. Most of these are tagged with a postcode of the headquarters of the company I’m dealing with. However, my next development step will be to give my back-end server code access to my inbox and allow it to scrape for online transaction receipts. This should reduce the false-positive rate down to something vanishingly small, and I should be able to deal with those left over with some sort of machine learning. After all, there’s a human-readable string attached to each transaction that details the retailer and sometimes other useful information.

A thought to ponder

A thought to ponder in the dead of night: In the near future, the absence of data is going to be increasingly unusual. If you think the data exhaust you leave behind yourself is wide and varied, then just you wait, because we’re at the banging-the-rocks-together stage right now.

If your data exhaust becomes assumed, what happens if you turn your phone off for an hour or two one night? What if you’re accused of a murder during that time period, and you can’t prove where you were? Perhaps in the future that’s going to be sufficiently unusual that it’s automatically suspicious. Innocent until proven guilty may underlie our current legal system, but that’s because our current legal system was codified in a very different era, one that was data poor rather than data rich. Perhaps in the future, the absence of data will imply guilt.

I discussed hidden data in this interview at Strata CA 2012.

Related:

• Got an iPhone or 3G iPad? Apple is recording your moves (April 2011)
• The Meat to Math ratio
• What your inbox knows
• Big Data: An opportunity in search of a metaphor

If things aren’t going to change very much on the surface, underneath the glass things might not be much different either. Oh, the devices will be faster, and they’ll have more cores, better displays, faster network connections, and the batteries will last longer. But fundamentally, they’ll still be the same. The device won’t provide you with any new levers on the world. With the exception of NFC, which admittedly is a big exception, there are no new sensory modalities on the horizon that are likely to be integrated into handsets. You’ll interact with your smartphone tomorrow in much the same way you interact with it today, at least in the near term.

That said, it’s quite possible that your smartphone will interact with the world in a very different way. That’s because the next mobile war has already begun, and you’ve seen nothing yet.

The phoney war

It began quietly, with little noise or fanfare, just over two years ago with Apple’s announcement of iOS 3, the External Accessory Framework, and the opportunity for partners in the MFi program to build external hardware that connected directly to the iPhone.

For the first time, it was easy, at least for certain values of easy, to build sensor hardware that connected to a mass-market mobile device. And for the first time, the mobile device had enough computing power and screen real estate to do something interesting with the sensor data.

Except of course, it wasn’t easy. While initially the External Accessory Framework was seen as having the potential to open up Apple’s platform to a host of external hardware and sensors, little of the innovation people were expecting actually occurred. Much of the blame was laid squarely at the feet of Apple’s own MFi program.

There was some headway made using the devices as sensor gateways, mainly in the medical community, which Apple had initially pushed heavily during the launch. But in the end, the framework was used to support a fairly predictable range of audio and video accessories from big-name manufacturers — although more recently there have been a few notable exceptions.

Opening a second front

Things stayed quiet until earlier this year when Google announced the Android Accessory Development Kit (ADK) at Google I/O in May.

While there was a lot of criticism of Google’s approach, it was justifiably hailed as a disruptive move by Google in what had become a fairly stagnant accessories market. Philip Torrone hit the right note when he speculated that this might mean the end of Apple’s restrictive MFi program.

I’ve talked about the Arduino here before. It allows rapid, cheap prototyping for embedded systems. Making Android the default platform for development of novel hardware was a brilliant move by Google. Maybe just a little too brilliant.

The counterattack by Apple

Around the middle of the year, right in the middle of Apple’s WWDC conference, I was approached by Redpark and sworn to secrecy. Apple was on the brink of approving a serial cable for iOS that they would let Redpark sell into the hobbyist market.

I’d known about the existence of the cable since the preceding November with the release of the SkyWire telescope control kit. I’d begged Redpark for developer access to their cable, and after signing a thick stack of NDAs, I got my hands on one around mid-December. At the time there seemed little chance of Apple ever approving the cable except for specific use cases where the cable and an accompanying iOS application were approved together as part of the MFi program — exactly as Apple had for Skywire for telescopes and Cisco had for networking gear.

The news that the cable might soon be generally available to hobbyists was surprising. Despite Apple’s beginnings — and the large community of indie developers surrounding its products — the hobbyist market isn’t something Apple is known for caring about these days. Quite the opposite: Apple is notorious for keeping its products as closed as possible.

Controlling an Arduino with an iPhone.

Close on the heels of Google’s ADK announcement, Apple’s sudden enthusiasm was suspiciously timed. Someone high up at Apple had obviously realized the disruptive nature of the ADK and this was their response, their counter-attack. Despite the Android ADK actually being an Arduino, it was now easier to talk to an Arduino from iOS using Redpark’s cable than it was to talk to an Arduino from Android.

The long war

The Android ADK board is only now appearing in large numbers as the open hardware community gears up to produce compatible boards cheaper than Google’s ruinously expensive initial batch of “official” developer boards. The Redpark cable also faced supply issues, with the initial production run selling out on the Maker Shed within a few days. We’re only now seeing it in larger volumes. So, despite appearances, it’s still the early days.

Discussing the Redpark cable at OSCON 2011.

I think the availability of both these products is going to prove to be amazingly disruptive in the longer term. After spending two days at the recent World Maker Faire in New York, I know there’s a lot of enthusiasm inside the Maker community for that disruption — and Apple may have the edge.

Because of Apple’s policy restrictions, you can only develop applications that work with Redpark’s cable for your own personal use or for distribution inside an enterprise environment without going through the MFi program. The ease of use and popularity of the iOS platform with developers means there will still be a big uptake, and after a few people struggle through the process, I think that, with time, the cable will spell the end of the MFi program.

Over the next couple of years, we’ll be seeing some real innovation in the external accessory product space. Rapid prototyping combined with ease of access to increasingly powerful mobile platforms means that the next mobile war, and the next big thing of a real ubiquitous computing environment, is just around the corner.

Related:

• Arduino is a building block for the world to come
• The next, next big thing
• Google I/O 2011: The Accessory Development Kit is a big deal
• Why Google Choosing Arduino Matters
• Parallel programming, Arduino and the good kind of trouble

Apple will join Google and Amazon as a major player in cloud computing. The 200 million iTunes users Apple brings with them puts the company on the same level as those other platforms. Despite that, the three companies obviously see the cloud in very different ways, and as a result have very different strategies.

Amazon is the odd man out. Their cloud offering is bare metal, contrasting sharply with Google, and now Apple’s, document-based model. To be fair, Amazon’s target market is very different, with their focus on service providers. If you’re a Valley start-up looking for storage and servers, you need look no further than Amazon’s Web Services platform.

Google and Apple’s document model contrasts sharply with Amazon’s service-stack approach. Both Google and Apple have attempted to abstract away things, like the file system, which stand between the end user and their data. An unsurprising difference perhaps, Google and Apple are consumer-facing companies that are marketing to the final end user rather than the people and companies who aim to provide services for those users.

But that’s where the similarity between Google and Apple breaks down. Google sees the cloud as a way to deprecate general purpose computers in the hands of their users. In the same way that their new Chromium OS is built for the web, their cloud strategy is an attempt to move Google’s users away from native applications so that their applications and data live in Google’s cloud of services. Perhaps coincidentally, this also gives Google the chance to display and target their advertising even more cleverly.

Apple’s approach is almost entirely the opposite. They see the cloud as a way to keep the general purpose computer on life support for a few more years until touch-based hardware is really ready to take over. Apple’s new cloud platform is built for native applications, in an attempt to pull users into native apps designed for their platforms. This method also gives Apple the chance to sell hardware, applications, and content that will lock users into their platform even more firmly. This is the basis of the often remarked “halo effect.”

At least on the surface things seem to be simple — the “why” of the thing is not in question. However it’s what hasn’t been said, at least openly, that raises the most interesting questions.

Web 2.0 Summit, being held October 17-19 in San Francisco, will examine “The Data Frame” — focusing on the impact of data in today’s networked economy.

Save $300 on registration with the code RADAR

Apple is fundamentally platform orientated. It’s deep in their company genetics. The ill-fated official cloning program from the mid-’90s, which was brought to a screeching halt by the return of Steve Jobs, seems to have set a deep fear inside the company about letting someone else control anything that might stand between the company and direct access to their customers.

At least to me, nothing confirms that mindset more than Apple’s return to designing their own processors in-house in Cupertino. Apple has a long history of using its own custom silicon, but it’s been more than five years since Apple has done so. With the move to Intel, the hope was to delegate nearly all of Apple’s custom chip development. Unfortunately, that proved to be a stumbling block when Apple built the first generation iPhone. The Samsung H1 processor in the original model wasn’t quite what Apple wanted, even though it was what had been asked for, and I think the return to custom silicon probably brought a sigh of relief in some corners of the company.

The link between custom chips and the cloud may seem tenuous at first glance, but I think Apple’s return to designing their own silicon is telling. Almost as telling as spending half a billion dollars on a custom data center to support their new iCloud service. Both moves show the company is now committed more than ever to controlling the verticals. From the chips inside the devices to the data centers their customers’ data ultimately resides on, Apple is committed to controlling the user experience, and the web has no place in that.

You might argue that this is because the web is “too open” and that threatens Apple’s platform. However, the continuing argument over openness, or lack there of, isn’t really relevant. Despite Google’s protestations to the contrary, neither of these two companies is particularly open. The very document-based model they’re both advocating in their cloud architectures precludes a truly open system. It’s such an obvious straw man argument that it’s not actually that interesting.

What is interesting is that there was little or no mention of the web, or HTML5, during Apple’s
WWDC keynote. I think you’ll see far less emphasis on HTML5 from Apple in the future, unless someone asks to do something with Apple’s platform the company disapproves of, and then the traditional answer of “Well, you can
always do that in HTML5″ will be rolled out again.

Apple has finally put their cards on the table. They have not yet bet the company on iCloud, but it’s telling how deep the integration into both iOS and OS X appears to be. They have for too much invested in iCloud for it to fail, if only in reputation. Whether the first incarnation lives up to its promises out of the box is still to be seen, but success isn’t out of the question. Despite MobileMe, Apple does know how to build large-scale reliable backend services. You only have to look at the App Store itself for an example.

So in the future don’t be too surprised to see Apple integrate iCloud even more tightly with both iOS and OS X. For the same strategic reasons, don’t be shocked to see more custom chips appear — I expect to see the arrival of ARM-based MacBooks and the transition away from Intel for Apple’s laptops. That’s because for Apple, It’s all about the platform.

Related:

• Four core takeaways from Apple’s WWDC keynote
• iClouds on the horizon

Those that fail to learn from history are doomed to repeat it, and the smart young things need to pay more attention. Because the trends underlying today’s computing should be evident to anyone with a sufficiently good grasp of computing history.

Depending on the state of technology, the computer industry oscillates between thin- and thick-client architectures. Either the bulk of our compute power and storage is hidden away in racks of (sometimes distant) servers, or alternatively, into a mass of distributed systems closer to home. This year’s reinvention of the mainframe is called cloud computing. While I’m a big supporter of cloud architectures, at least at the moment, I’ll be interested to see those preaching it as a last and final solution of all our problems proved wrong, yet again, when computing power catches up to demand once more and you can fit today’s data center inside a box not much bigger than a cell phone.

Thinking that just couldn’t happen? You should think again, because it already has. The iPad 2 beats most super computers from the early ’90s in raw compute power, and it would have been on the world-wide top 500 list of super computers well into 1994. There isn’t any reason to suspect that, at least for now, that sort of trend isn’t going to continue.

Yesterday’s next big thing

Yesterday’s “next big thing” was the World Wide Web. I still vividly remember standing in a draughty computing lab, almost 20 years ago now, looking over the shoulder of someone who had just downloaded first public build of NCSA Mosaic via some torturous method. I shook my head and said “It’ll never catch on, why would you want images?” That shows what I know. Although to be fair, I was a lot younger back then. I was failing to grasp history because I was neither well read enough, nor old enough, to have seen it all before. And since I still don’t claim to be either well read or old enough this time around, perhaps you should take everything I’m saying with a pinch of salt. That’s the thing with the next big thing: it’s always open to interpretation.

The next big thing?

The machines we grew up with are yesterday’s news. They’re quickly being replaced by consumption devices, with most of the rest of day-to-day computing moving into the environment and becoming embedded into people’s lives. This will happen almost certainly
without people noticing.

While it’s pretty obvious that mobile is the current “next” big thing, it’s arguable whether mobile itself has already peaked. The sleek lines of the iPhone in your pocket are already almost as dated as the beige tower that used to sit next to the CRT on your desk.

Technology has not quite caught up to the overall vision and neither have we — we’ve been trying to reinvent the desktop computer in a smaller form factor. That’s why the mobile platforms we see today are just stepping stones.

Most people just want gadgets that work, and that do the things they want them to do. People never really wanted computers. They wanted what computers could do for them. The general purpose machines we think of today as “computers” will naturally dissipate out into the environment as our technology gets better.

The next, next big thing

To those preaching cloud computing and web applications as the next big thing: they’ve already had their day and the web as we know it is a dead man walking. Looking at the job board at O’Reilly’s Strata conference earlier in the year, the next big thing is obvious. It’s data. Heck, it’s not even the next big thing anymore. It’s pulling into the station, and to data scientists, the web and its architecture is just a commodity. Bought and sold in bulk.

The overflowing job board at February’s Strata conference.

As for the next, next big thing? Ubiquitous computing is the thing after the next big thing, and almost inevitably the thirst for more data will drive it. But then eventually, inevitably, the data will become secondary — a commodity. Yesterday’s hot job was a developer, today with the arrival of Big Data it has become a mathematician. Tomorrow it could well be a hardware hacker.

Count on it. History goes in cycles and only the names change.

Related:

• Arduino is a building block for the world to come.
• The abandonment of technology
• The return of the Personal Area Network

Every so often a piece of technology can become a lever that lets people move the world, just a little bit. The Arduino is one of those levers.

It started off as a project to give artists access to embedded micro-processors for interaction design projects, but I think it’s going to end up in a museum as one of the building blocks of the modern world. It allows rapid, cheap, prototyping for embedded systems. It turns what used to be fairly tough hardware problems into simpler software problems.

The Arduino, and the open hardware movement that has grown up with it, and at least to certain extent around it, is enabling a generation of high-tech tinkerers both to break the seals on proprietary technology, and prototype new ideas with fairly minimal hardware knowledge. This maker renaissance has led to an interesting growth in innovation. People aren’t just having ideas, they’re doing something with them.

Goodbye desktop

The underlying trend is clear. The general purpose computer is a dead end. Most people just want gadgets that work, and that do the things they want them to do. They never really wanted computers. They wanted what computers could do for them.

While general purpose computers will live on, like the horse after the arrival of the automobile, these systems will be relegated to two small niches. Those of us that build the embedded systems people are using elsewhere will still have a need for general purpose computers, as will those who can’t resist tinkering. But that’s the extent of it. Nobody else will need them. Quite frankly, nobody else will want them.

The humble Arduino is the start of that. The board has multiple-form factors, but a single-programming interface. Sizes range from the “standard” palm of your hand for prototyping, down to the size of your thumb for the almost-professional almost-products now starting to come out of the maker renaissance. Arduino, and its relatives, will be part of everything from wearable versions like the Lilypad, sized and customized to be stitched into clothing, to mobile phone hardware accessories, to specially built boards launched into space on the new generation of nano-satellites built on a shoe-string budget by hobbyists.

Every interesting hardware prototype to come along seems to boast that it is Arduino-compatible, or just plain built on top of an Arduino. It’s everywhere.

Maker Faire Bay Area will be held May 21-22 in San Mateo, Calif. Event details, exhibitor profiles, and ticket information can be found at the Maker Faire site.

Things are still open. They’re just different things.

There has been a great deal of fear-mongering about the demise of the general purpose computer and the emergence of a new generation of consumption devices as more-or-less closed platforms. When the iPad made its debut, Cory Doctorow argued that closed platforms send the wrong signal:

Buying an iPad for your kids isn’t a means of jump-starting the realization that the world is yours to take apart and reassemble; it’s a way of telling your offspring that even changing the batteries is something you have to leave to the professionals.

I’m philosophical about the passing of the computer. What we’re seeing here is a transition from one model of computing to another. We’ve seen that before and there were similar outcries for the death of the mainframe, as there has been for the death of the desktop. There is plenty of room for closed platforms, but the underlying trend is toward more openness, not less. It’s just the things that are open and the things that are closed are changing. The skills needed to work with the technology are changing as well.

What the Arduino and the open hardware movement have done is made hard things easy, and impossible things merely hard. Before now, getting to the prototype stage for a hardware project was hard, at least for most people, and going beyond a crude prototype was impossible for many. Now it’s the next big thing.

Related:

• Why Google Choosing Arduino Matters
• Parallel programming, Arduino and the good kind of trouble
• What you can do with Processing and Arduino