The unwelcome guest: Why VMs aren’t the solution for next-gen applications

Scale-out applications need scaled-in virtualization.

scale_in_esterno_Mia_Felicita_Bertelli_FlickrData center operating systems are emerging as a first-class category of distributed system software. Hadoop, for example, is evolving from a MapReduce framework into YARN, a generic platform for scale-out applications.

To enable a rich ecosystem of diverse applications to coexist on these platforms, providing adequate isolation is crucial. The isolation mechanism must enforce resource limits, decouple software dependencies among applications and the host, provide security and privacy, confine failures, etc. Containers offer a simple and elegant solution to the problem. However, a question that comes up frequently is: Why not virtual machines (VMs)? After all, these systems face a number of the same challenges that have been solved by virtualization for traditional enterprise applications.

All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections” — David Wheeler

Read more…


On the evolution of machine learning

From linear models to neural networks: an interview with Reza Zadeh.

Get notified when our free report, “Future of Machine Intelligence: Perspectives from Leading Practitioners,” is available for download. The following interview is one of many that will be included in the report.

As part of our ongoing series of interviews surveying the frontiers of machine intelligence, I recently interviewed Reza Zadeh. Reza is a Consulting Professor in the Institute for Computational and Mathematical Engineering at Stanford University and a Technical Advisor to Databricks. His work focuses on Machine Learning Theory and Applications, Distributed Computing, and Discrete Applied Mathematics.

Key Takeaways

  • Neural networks have made a comeback and are playing a growing role in new approaches to machine learning.
  • The greatest successes are being achieved via a supervised approach leveraging established algorithms.
  • Spark is an especially well-suited environment for distributed machine learning.

David Beyer: Tell us a bit about your work at Stanford

Reza Zadeh: At Stanford, I designed and teach distributed algorithms and optimization (CME 323) as well as a course called discrete mathematics and algorithms (CME 305). In the discrete mathematics course, I teach algorithms from a completely theoretical perspective, meaning that it is not tied to any programming language or framework, and we fill up whiteboards with many theorems and their proofs. Read more…


Our future sits at the intersection of artificial intelligence and blockchain

The O'Reilly Radar Podcast: Steve Omohundro on AI, cryptocurrencies, and ensuring a safe future for humanity.


Subscribe to the O’Reilly Radar Podcast to track the technologies and people that will shape our world in the years to come.

I met up with Possibility Research president Steve Omohundro at our Bitcoin & the Blockchain Radar Summit to talk about an interesting intersection: artificial intelligence (AI) and blockchain/cryptocurrency technologies. This Radar Podcast episode features our discussion about the role cryptocurrency and blockchain technologies will play in the future of AI, Omohundro’s Self Aware Systems project that aims to ensure intelligent technologies are beneficial for humanity, and his work on the Pebble cryptocurrency.

Synthesizing AI and crypto-technologies

Bitcoin piqued Omohundro’s interest from the very start, but his excitement built as he started realizing the disruptive potential of the technology beyond currency — especially the potential for smart contracts. He began seeing ways the technology will intersect with artificial intelligence, the area of focus for much of his work:

I’m very excited about what’s happening with the cryptocurrencies, particularly Ethereum. I would say Ethereum is the most advanced of the smart contracting ideas, and there’s just a flurry of insights, and people are coming up every week with, ‘Oh we could use it to do this.’ We could have totally autonomous corporations running on the blockchain that copy what Uber does, but much more cheaply. It’s like, ‘Whoa what would that do?’

I think we’re in a period of exploration and excitement in that field, and it’s going to merge with the AI systems because programs running on the blockchain have to connect to the real world. You need to have sensors and actuators that are intelligent, have knowledge about the world, in order to integrate them with the smart contracts on the blockchain. I see a synthesis of AI and cryptocurrencies and crypto-technologies and smart contracts. I see them all coming together in the next couple of years.

Read more…


Smell and taste: New frontiers for experience design

Designers need not start from scratch as they wrestle with orchestrating experiences that span digital and physical.


Download a free copy of Designing for the Internet of Things, a curated collection of chapters from the O’Reilly Design library. This post is an excerpt from Understanding Industrial Design, by Simon King and Kuen Chang, one of the books included in the curated collection.

Two of our richest senses, smell and taste, are not often associated with design. However, the creation of objects that support these senses is an ancient practice, embodied best by the tea set, where rituals of assembly and service lead to hints of the aroma. Holding the tea cup warms your hand without burning it, and the slow sipping of the tea forms a communal bond with other participants. Outside of classic and common serving items, designers today are increasingly finding new ways to collaborate with chefs and food companies to design with smell and taste in mind, forging a new frontier for sensorial design.

Martin Kastner is the founder and principal of Crucial Detail, a studio in Chicago that specializes in custom pieces to support unique culinary experiences. Martin is best known for his work designing serviceware concepts for Alinea, the 3-star Michelin restaurant founded by chef Grant Achatz. That collaboration has extended to other restaurants owned by Achatz, including The Aviary, a cocktail bar that prides itself on serving drinks with the same level of attention as a fine dinner.

At The Aviary, one of the most popular creations by Crucial Detail is the Porthole Infuser, a round vessel that presents the ingredients of a patron’s cocktail between two flat panes of glass, emphasizing the transformative action of the steeping process and building anticipation for the cocktail’s taste. The Porthole Infuser takes a part of the preparation process that is normally hidden and brings it directly to the person’s table, providing time for the drinker to contemplate the ingredients on display, creating a mental checklist for their tongue to seek out when they take their first sip.

The popularity of the Porthole Infuser at the Aviary led Kastner to create a Kickstarter campaign to fund the additional design and manufacturing required to release it as a commercial product. Support for the project was dramatic, achieving 25 times more funding than originally asked. This backing set the course for a redesign that allowed the infuser to be manufactured at scale and sold for $100, down from the several hundred dollars that each custom constructed version for The Aviary cost.

The Porthole Infuser is marketed as more than a cocktail tool, working equally well to support the smell and taste of oils, teas, or any other infusion recipe. It’s an example of how designers can enhance the dining experience, not by crafting the smell or taste of the food itself, but working in collaboration with a chef to heighten our awareness of those senses. Read more…


More tools for managing and reproducing complex data projects

A survey of the landscape shows the types of tools remain the same, but interfaces continue to improve.


As data projects become complex and as data teams grow in size, individuals and organizations need tools to efficiently manage data projects. A while back, I wrote a post on common options, and I closed that piece by asking:

Are there completely different ways of thinking about reproducibility, lineage, sharing, and collaboration in the data science and engineering context?

At the time, I listed categories that seemed to capture much of what I was seeing in practice: (proprietary) workbooks aimed at business analysts, sophisticated IDEs, notebooks (for mixing text, code, and graphics), and workflow tools. At a high level, these tools aspire to enable data teams to do the following:

  • Reproduce their work — so they can rerun and/or audit when needed
  • Collaborate
  • Facilitate storytelling — because in many cases, it’s important to explain to others how results were derived
  • Operationalize successful and well-tested pipelines — particularly when deploying to production is a long-term objective

As I survey the landscape, the types of tools remain the same, but interfaces continue to improve, and domain specific languages (DSLs) are starting to appear in the context of data projects. One interesting trend is that popular user interface models are being adapted to different sets of data professionals (e.g. workflow tools for business users). Read more…


Empathy: The designer’s superpower

Scott Jenson on empathy, interaction on demand, and Google’s Physical Web Project.

Toolbox_florianric_FlickrI recently connected with veteran designer Scott Jenson, who is currently developing the Physical Web Project with the Chrome team at Google. We’ve been talking quite a bit about empathy in the past few months here at O’Reilly, and Scott’s recent blog post, The Paradox of Empathy, caught my attention. I sat down with him to learn more about his thinking around empathy and to talk about his work on the Physical Web Project.

Empathy is part of every great designer’s toolkit

Jenson is often asked for recommendations for learning the next tool, or program. but as he explains, learning how to empathize is fundamental to product design:

When I reflected on what I wanted people to understand, what the core thing was, it wasn’t a technique. It wasn’t a visual style. It wasn’t learning a certain program. The core thing was making sure that you never thought about the product from your point of view, but from somebody else’s point of view. That’s what prompted the [The Paradox of Empathy] post.

He breaks empathy down into four components:

I basically take the whole design process from soup to nuts and break it up into four types of things, what I called understanding, bridging, flowing, and refining, which is a little bit of wordplay, but it was just really trying to say that most people talk about the icons and the buttons. That’s the last category, the refining. What I tried to do was to go back in time to get earlier and earlier interactions with people. So, the flowing is basically just how the whole program feels and what metaphors do you use, and how many steps do they take. It’s the level above the bits. Bridging was about matching the technology to the actual user needs. The most important one, the one that we actually tried to do the most when I was at Frog Design, was understanding, which was just to understand what people were doing, what were they up to, where they were at. In fact, to the point where you’re not even designing a product for them. One of the reasons why I think [The Paradox of Empathy] post got some positive response, was the fact that the first two were so clearly focused on user research.

Read more…