Four short links: 6 May 2015

Self-Driving Cars, Cloud BigTable, Define "Uptime", and Continuous Delivery Architectures

  1. Andrew Ng (Wired) — I think self-driving cars are a little further out than most people think. There’s a debate about which one of two universes we’re in. In the first universe it’s an incremental path to self-driving cars, meaning you have cruise control, adaptive cruise control, then self-driving cars only on the highways, and you keep adding stuff until 20 years from now you have a self-driving car. In universe two you have one organization, maybe Carnegie Mellon or Google, that invents a self-driving car and bam! You have self-driving cars. It wasn’t available Tuesday but it’s on sale on Wednesday. I’m in universe one. I think there’s a lot of confusion about how easy it is to do self-driving cars. There’s a big difference between being able to drive a thousand miles, versus being able to drive anywhere. And it turns out that machine-learning technology is good at pushing performance from 90 to 99 percent accuracy. But it’s challenging to get to four nines (99.99 percent). I’ll give you this: we’re firmly on our way to being safer than a drunk driver.
  2. Google Cloud BigTable — Google’s BigTable, with Apache HBase API, single-digit millisecond latency, and “fully managed”. G are hell-bent on catching up with Amazon and Microsoft at this cloud serving thing.
  3. Call Me Maybe: AerospikeWe’re setting a timeout of 500ms here, and operations still time out every time a partition between nodes occurs. In these tests we aren’t interfering with client-server traffic at all. Aerospike may claim “100% uptime”, but this is only meaningful with respect to particular latency bounds. Given Aerospike claims millisecond-scale latencies, you may want to reconsider whether you consider this “uptime”.
  4. 31 Continuous Delivery Architectures (Slideshare) — from a vendor, so one name crops up repeatedly (other than “Jenkins”), but it’s still good devops voyeurism/envy.

Cottage industry 2.0

The O'Reilly Solid Podcast: Amanda Peyton of Etsy on the rise of craft.

Our new episode of the Solid Podcast brings us to Etsy, where David and I spoke with Amanda Peyton, a serial entrepreneur and product manager at Etsy, about the company’s role as a macro-community of micro-communities.

The connection between Etsy and Solid might not be obvious at first. Etsy’s specialty is the ultra-analog: handmade crafts that represent a return to an earlier era of artisan design and manufacturing.

But Etsy is emblematic of how Web platforms have transformed the relationship between product creators and product consumers. It offers rapid feedback from the market, quick discovery of new trends, and access to a large and diverse enough customer base that even extraordinarily niche products can be viable.

The result is a community of distributed manufacturers that’s responsive and efficient. For goods that can be made without a large, well-capitalized factory (even some electronics now fall into this category), Etsy may be the future of manufacturing. Read more…

Four short links: 5 May 2015

Four short links: 5 May 2015

Agile Hardware, Time Series Data, Data Loss, and Automating Security

  1. How We Do Agile Hardware Development at MeldIn every sprint we built both hardware and software. This doesn’t mean we had a fully fabricated new board rev once a week. […] We couldn’t build a complete new board every week, and early on we didn’t even know for sure what parts we wanted in our final BOM (Bill of Materials) so we used eval boards. These stories of how companies iterated fast will eventually build a set of best practices for hardware startups, similar to those in software.
  2. Recording Time Series — if data arrives with variable latency, timestamps are really probabilistic ranges. How do you store your data for searches and calculations that reflect reality, and are not erroneous because you’re ignoring a simplification you made to store the data more conveniently?
  3. Call Me Maybe, ElasticSearch 1.5.0To be precise, Elasticsearch’s transaction log does not put your data safety first. It puts it anywhere from zero to five seconds later. In this test we kill random Elasticsearch processes with kill -9 and restart them. In a datastore like Zookeeper, Postgres, BerkeleyDB, SQLite, or MySQL, this is safe: transactions are written to the transaction log and fsynced before acknowledgement. In Mongo, the fsync flags ensure this property as well. In Elasticsearch, write acknowledgement takes place before the transaction is flushed to disk, which means you can lose up to five seconds of writes by default. In this particular run, ES lost about 10% of acknowledged writes.
  4. FIDO — Netflix’s open source system for automatically analyzing security events and responding to security incidents.

The unwelcome guest: Why VMs aren’t the solution for next-gen applications

Scale-out applications need scaled-in virtualization.

scale_in_esterno_Mia_Felicita_Bertelli_FlickrData center operating systems are emerging as a first-class category of distributed system software. Hadoop, for example, is evolving from a MapReduce framework into YARN, a generic platform for scale-out applications.

To enable a rich ecosystem of diverse applications to coexist on these platforms, providing adequate isolation is crucial. The isolation mechanism must enforce resource limits, decouple software dependencies among applications and the host, provide security and privacy, confine failures, etc. Containers offer a simple and elegant solution to the problem. However, a question that comes up frequently is: Why not virtual machines (VMs)? After all, these systems face a number of the same challenges that have been solved by virtualization for traditional enterprise applications.

All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections” — David Wheeler

Read more…


On the evolution of machine learning

From linear models to neural networks: an interview with Reza Zadeh.

Get notified when our free report, Future of Machine Intelligence: Perspectives from Leading Practitioners, is available for download. The following interview is one of many that will be included in the report.

As part of our ongoing series of interviews surveying the frontiers of machine intelligence, I recently interviewed Reza Zadeh. Reza is a Consulting Professor in the Institute for Computational and Mathematical Engineering at Stanford University and a Technical Advisor to Databricks. His work focuses on Machine Learning Theory and Applications, Distributed Computing, and Discrete Applied Mathematics.

Key Takeaways

  • Neural networks have made a comeback and are playing a growing role in new approaches to machine learning.
  • The greatest successes are being achieved via a supervised approach leveraging established algorithms.
  • Spark is an especially well-suited environment for distributed machine learning.

David Beyer: Tell us a bit about your work at Stanford

Reza Zadeh: At Stanford, I designed and teach distributed algorithms and optimization (CME 323) as well as a course called discrete mathematics and algorithms (CME 305). In the discrete mathematics course, I teach algorithms from a completely theoretical perspective, meaning that it is not tied to any programming language or framework, and we fill up whiteboards with many theorems and their proofs. Read more…


Our future sits at the intersection of artificial intelligence and blockchain

The O'Reilly Radar Podcast: Steve Omohundro on AI, cryptocurrencies, and ensuring a safe future for humanity.


Subscribe to the O’Reilly Radar Podcast to track the technologies and people that will shape our world in the years to come.

I met up with Possibility Research president Steve Omohundro at our Bitcoin & the Blockchain Radar Summit to talk about an interesting intersection: artificial intelligence (AI) and blockchain/cryptocurrency technologies. This Radar Podcast episode features our discussion about the role cryptocurrency and blockchain technologies will play in the future of AI, Omohundro’s Self Aware Systems project that aims to ensure intelligent technologies are beneficial for humanity, and his work on the Pebble cryptocurrency.

Synthesizing AI and crypto-technologies

Bitcoin piqued Omohundro’s interest from the very start, but his excitement built as he started realizing the disruptive potential of the technology beyond currency — especially the potential for smart contracts. He began seeing ways the technology will intersect with artificial intelligence, the area of focus for much of his work:

I’m very excited about what’s happening with the cryptocurrencies, particularly Ethereum. I would say Ethereum is the most advanced of the smart contracting ideas, and there’s just a flurry of insights, and people are coming up every week with, ‘Oh we could use it to do this.’ We could have totally autonomous corporations running on the blockchain that copy what Uber does, but much more cheaply. It’s like, ‘Whoa what would that do?’

I think we’re in a period of exploration and excitement in that field, and it’s going to merge with the AI systems because programs running on the blockchain have to connect to the real world. You need to have sensors and actuators that are intelligent, have knowledge about the world, in order to integrate them with the smart contracts on the blockchain. I see a synthesis of AI and cryptocurrencies and crypto-technologies and smart contracts. I see them all coming together in the next couple of years.

Read more…

Comment: 1