Nat has chaired the O'Reilly Open Source Convention and other O'Reilly conferences for over a decade. He ran the first web server in New Zealand, co-wrote the best-selling Perl Cookbook, and was one of the founding Radar bloggers. He lives in New Zealand and consults in the Asia-Pacific region.
Shadow Robot’s Dextrous Hand (Robohub) — The now infamous crab bisque recipe was prepared copying the exact movements of 2011 BBC MasterChef winner Tim Anderson. The robot switches on a hob, scrapes butter into the pan and adds prepared and measured ingredients – from a specific place on the worktop. But, it doesn’t do the chopping… That, says Rich, is an AI problem.
Predicting a Billboard Music Hit (YouTube) — Shazam VP of Music and Platforms at Strata London. With relative accuracy, we can predict 33 days out what song will go to No. 1 on the Billboard charts in the U.S.
What Two Programmers Have Revealed So Far About Seattle Police Officers Who Are Still in Uniform — through their shrewd use of Washington’s Public Records Act, the two Seattle residents are now the closest thing the city has to a civilian police-oversight board. In the last year and a half, they have acquired hundreds of reports, videos, and 911 calls related to the Seattle Police Department’s internal investigations of officer misconduct between 2010 and 2013. And though they have only combed through a small portion of the data, they say they have found several instances of officers appearing to lie, use racist language, and use excessive force—with no consequences. In fact, they believe that the Office of Professional Accountability (OPA) has systematically “run interference” for cops. In the aforementioned cases of alleged officer misconduct, all of the involved officers were exonerated and still remain on the force.
Andrew Ng (Wired) — I think self-driving cars are a little further out than most people think. There’s a debate about which one of two universes we’re in. In the first universe it’s an incremental path to self-driving cars, meaning you have cruise control, adaptive cruise control, then self-driving cars only on the highways, and you keep adding stuff until 20 years from now you have a self-driving car. In universe two you have one organization, maybe Carnegie Mellon or Google, that invents a self-driving car and bam! You have self-driving cars. It wasn’t available Tuesday but it’s on sale on Wednesday. I’m in universe one. I think there’s a lot of confusion about how easy it is to do self-driving cars. There’s a big difference between being able to drive a thousand miles, versus being able to drive anywhere. And it turns out that machine-learning technology is good at pushing performance from 90 to 99 percent accuracy. But it’s challenging to get to four nines (99.99 percent). I’ll give you this: we’re firmly on our way to being safer than a drunk driver.
Google Cloud BigTable — Google’s BigTable, with Apache HBase API, single-digit millisecond latency, and “fully managed”. G are hell-bent on catching up with Amazon and Microsoft at this cloud serving thing.
Call Me Maybe: Aerospike — We’re setting a timeout of 500ms here, and operations still time out every time a partition between nodes occurs. In these tests we aren’t interfering with client-server traffic at all. Aerospike may claim “100% uptime”, but this is only meaningful with respect to particular latency bounds. Given Aerospike claims millisecond-scale latencies, you may want to reconsider whether you consider this “uptime”.
How We Do Agile Hardware Development at Meld — In every sprint we built both hardware and software. This doesn’t mean we had a fully fabricated new board rev once a week. […] We couldn’t build a complete new board every week, and early on we didn’t even know for sure what parts we wanted in our final BOM (Bill of Materials) so we used eval boards. These stories of how companies iterated fast will eventually build a set of best practices for hardware startups, similar to those in software.
Recording Time Series — if data arrives with variable latency, timestamps are really probabilistic ranges. How do you store your data for searches and calculations that reflect reality, and are not erroneous because you’re ignoring a simplification you made to store the data more conveniently?
Call Me Maybe, ElasticSearch 1.5.0 — To be precise, Elasticsearch’s transaction log does not put your data safety first. It puts it anywhere from zero to five seconds later. In this test we kill random Elasticsearch processes with kill -9 and restart them. In a datastore like Zookeeper, Postgres, BerkeleyDB, SQLite, or MySQL, this is safe: transactions are written to the transaction log and fsynced before acknowledgement. In Mongo, the fsync flags ensure this property as well. In Elasticsearch, write acknowledgement takes place before the transaction is flushed to disk, which means you can lose up to five seconds of writes by default. In this particular run, ES lost about 10% of acknowledged writes.
FIDO — Netflix’s open source system for automatically analyzing security events and responding to security incidents.
Silicon Valley Primer — a short but interesting precis of what made the Valley great, with stories of the nobility. From a historian. All these new people pouring into what had been an agricultural region meant that it was possible to create a business environment around the needs of new companies coming up, rather than adapting an existing business culture to accommodate the new industries. In what would become a self-perpetuating cycle, everything from specialized law firms, recruiting operations and prototyping facilities; to liberal stock option plans; to zoning laws; to community college course offerings developed to support a tech-based business infrastructure.
Introduction to GraphQL — We believe that GraphQL represents a novel way of structuring the client-server contract. Servers publish a type system specific to their application, and GraphQL provides a unified language to query data within the constraints of that type system. That language allows product developers to express data requirements in a form natural to them: a declarative and hierarchal one. The nightmare of the ad hoc API morass is a familiar one …
Critical Steps to Building First Quantum Computer — The IBM breakthroughs, described in the April 29 issue of the journal Nature Communications, show for the first time the ability to detect and measure the two types of quantum errors (bit-flip and phase-flip) that will occur in any real quantum computer. Until now, it was only possible to address one type of quantum error or the other, but never both at the same time. This is a necessary step toward quantum error correction, which is a critical requirement for building a practical and reliable large-scale quantum computer.
Five Short Stories About the Life and Times of Ideas (Nautilus) — In the following five short chapters, David Krakauer, an evolutionary theorist, and president elect of the Santa Fe Institute, haven of complex systems research, examines five facets of chain reactions, each typifying how ideas spread through science and culture. Together they tell a story of how the ideas that define humanity arise, when and why they die or are abandoned, the surprising possibilities for continued evolution, and our responsibility to nurture thought that might enlighten our future.
Go By Example — a chance to replicate the experience of learning Perl or PHP, whereby you know nothing but copy and adapt other people’s code until it works and you’ve empirically acquired an intuition for what will trigger the compiler’s deathray and eventually someone points you to the docs that were opaque and suddenly a lightbulb goes off in your head and you shout “omigod I finally get it!” and the Real Engineer beside you rolls their eyes and gets back to genericising their containers for consensus or whatever it is that Real Engineers do now.
Chinese Shopping for Robotic Ventures — Amazon has drones, Facebook has VR, Google and China are fighting it out for Robots. Meanwhile, Apple is curled up in a mountain filled with gold, their paws twitching and stroking their watches as they dream of battles to come.
Robot Arm Brings Humanity Back to the Stone Age (IEEE) — Using robots to build a massive database of scrape/wear patterns for different stone-age tools. Currently, Iovita is experiencing some opposition from within his own profession. Some believe that manual experiments are closer to the past reality; others find that use-wear analysis in general does not advance archaeological theory. Iovita thinks this is mainly due to the fact that most archaeologists have a humanities background and are not familiar with the world of engineers. OH SNAP.
More Tools for Managing and Reproducing Complex Data Projects (Ben Lorica) — As I survey the landscape, the types of tools remain the same, but interfaces continue to improve, and domain specific languages (DSLs) are starting to appear in the context of data projects. One interesting trend is that popular user interface models are being adapted to different sets of data professionals (e.g. workflow tools for business users).
Disinformation Visualisation: How to Lie with Datavis — We don’t spread visual lies by presenting false data. That would be lying. We lie by misrepresenting the data to tell the very specific story we’re interested in telling. If this is making you slightly uncomfortable, that’s a good thing; it should. If you’re concerned about adopting this new and scary habit, well, don’t worry; it’s not new. Just open your CV to be reminded you’ve lied with truthful data before. This time, however, it will be explicit and visual. (via Regine Debatty)
Microtugs — a new type of small robot that can apply orders of magnitude more force than it weighs. This is in stark contrast to previous small robots that have become progressively better at moving and sensing, but lacked the ability to change the world through the application of human-scale loads.
Vault — a tool for securely managing secrets and encrypting data in-transit.
iSAX: Indexing and Mining Terabyte Sized Time Series (PDF) — Our approach allows both fast exact search and ultra-fast approximate search. We show how to exploit the combination of both types of search as sub-routines in data mining algorithms, allowing for the exact mining of truly massive real-world data sets, containing millions of time series. (via Benjamin Black)
Defining Mobile (Luke Wroblewski) — numbers on size, orientation, and # of thumbs across mobile users. 94% of the time, it’s in portrait mode.
PwC Manufacturing Barometer: Robotics — Planned acquisition of robotics systems over the next two to three years was cited by a maximum of 58% -– with nearly one-third (31%) planning to acquire a moderate amount (25%) or many more robotics systems (only 6%). A larger number plan to acquire a limited number of robotics systems (27%). (via Robohub)
The Asshole Factory (Umair Haque) — The Great Enterprise of this age is the Asshole Industry. And that’s not just a tragedy. It is something approaching the moral equivalent of a crime. For it demolishes human potential in precisely the same way as locking up someone innocent, and throwing away the key.
‘Living Figures’ Make Their Debut (Nature) — In July last year, neurobiologist Björn Brembs published a paper about how fruit flies walk. Nine months on, his paper looks different: another group has fed its data into the article, altering one of the figures. The update — to figure 4 — marks the debut of what the paper’s London-based publisher, Faculty of 1000 (F1000), is calling a living figure, a concept that it hopes will catch on in other articles. Brembs, at the University of Regensburg in Germany, says that three other groups have so far agreed to add their data, using software he wrote that automatically redraws the figure as new data come in.
Strategies Against Architecture (Seb Chan and Aaron Straup Cope) — the story of the design of the Cooper Hewitt’s clever “pen,” which visitors to the design museum use to collect the info from their favourite exhibits. (Visit the Cooper Hewitt when you’re next in NYC; it’s magnificent.)
Two Way Street — an independent explorer for The British Museum collection, letting you browse by year acquired, year created, type of object, etc. I note there are more things from a place called “Brak” than there are from USA. Facets are awesome. (via Courtney Johnston)
The Saddest Moment (PDF) — “How can you make a reliable computer service?” the presenter will ask in an innocent voice before continuing, “It may be difficult if you can’t trust anything and the entire concept of happiness is a lie designed by unseen overlords of endless deceptive power.” The presenter never explicitly says that last part, but everybody understands what’s happening. Making distributed systems reliable is inherently impossible; we cling to Byzantine fault tolerance like Charlton Heston clings to his guns, hoping that a series of complex software protocols will somehow protect us from the oncoming storm of furious apes who have somehow learned how to wear pants and maliciously tamper with our network packets. Hilarious. (via Tracy Chou)