Startups suggest big data is moving to the clouds

A look at the winners from a showcase of some of the most innovative big data startups.

At Strata + Hadoop World in London last week, we hosted a showcase of some of the most innovative big data startups. Our judges narrowed the field to 10 finalists, from whom they — and attendees — picked three winners and an audience choice.

Underscoring many of these companies was the move from software to services. As industries mature, we see a move from custom consulting to software and, ultimately, to utilities — something Simon Wardley underscored in his Data Driven Business Day talk, and which was reinforced by the announcement of tools like Google’s Bigtable service offering.

This trend was front and center at the showcase:

  • Winner Modgen, for example, generates recommendations and predictions, offering machine learning as a cloud-based service.
  • While second-place Brytlyt offers their high-performance database as an on-premise product, their horizontally scaled-out architecture really shines when the infrastructure is elastic and cloud based.
  • Finally, third-place OpenSensors’ real-time IoT message platform scales to millions of messages a second, letting anyone spin up a network of connected devices.

Ultimately, big data gives clouds something to do. Distributed sensors need a widely available, connected repository into which to report; databases need to grow and shrink with demand; and predictive models can be tuned better when they learn from many data sets. Read more…

Comment

A vision of a decentralized IoT stack

The O'Reilly Radar Podcast: Eric Jennings on the importance of creating a decentralized Internet for the Internet of Things.

Crisscross_Jan_Jurewicz_Flickr

Subscribe to the O’Reilly Radar Podcast to track the technologies and people that will shape our world in the years to come.

In this week’s Radar Podcast, I followed up with Eric Jennings, co-founder and CEO of Filament, about his vision of a decentralized Internet. In last week’s episode, Jennings chatted with O’Reilly’s Mac Slocum a bit about a decentralized Internet in the context of the Internet of Things, and I ventured a bit deeper into the topic this week.

I asked Eric about the model — what would a decentralized Internet for the IoT look like and how would it work? He likened it to the Web:

We actually take a large portion of our model, our mental model, about a decentralized IoT from the early Web. If you imagine back in the early Web days — way back, mid-80s, early 90s — HTTP and websites had just started coming around, and they were originally focused and designed for academic research papers to link to each other.

Back then there was this entire, and there still is, there’s an entire open protocol stack that the Web runs on. Since any site could link directly to another site, it became very open and friendly, and there were all these wonderful things that emerged from — the Facebooks and Googles and WordPresses of the world were built on top of this standardized open reference platform.

What we like to think is, what would that look like if you took that concept and mapped it over onto the Internet of Things? What similar analogies to the Facebooks and Googles and WordPresses would we see if we had a truly decentralized and open IoT stack, and not necessarily one that’s full of silos and verticalized specific solutions to small industry segments.

Read more…

Comment

Finding new in the Web

Learning from the Fluent Conference.

At Fluent 2015, we brought together a variety of stories about front-end engineering – some technical, some social, some more intricately intertwined.

From the very first day, it was clear that React was the big technical story of the conference, taking the place that Angular (which is still clearly important!) had had the previous year. Tutorials and sessions were busy, and I kept hearing conversation about React. Sometimes it was “what is React supposed to do?” but other times people were talking about exciting corners of React Native or techniques for integrating React with a variety of frameworks.

React makes me happy because it solves the problem a lot of people didn’t quite realize they had. Suddenly they are very enthusiastic about stuff that used to be really annoying. The Document Object Model (DOM) has been the foundation of most of the interactive work on the web since 1998, but it wasn’t very much fun then. As developers really get deeper into these things, the DOM has not exactly been a crowd-pleaser. In some ways React is a wrapper for the DOM, and in many ways it’s a just a better way to interact with the document tree.

The other technical key this year was JavaScript, often specifically ECMAScript 6 (ES6), the latest release. Brendan Eich talked about a world in which compiling to JavaScript has become normal, and how that frees much of the future development of JavaScript and the Web. Even Dart, which many of us saw as an attempt to replace JavaScript, has a home in this world.
Read more…

Comment

Signals from Strata + Hadoop World 2015 in London

Key insights from Strata + Hadoop World 2015 in London.

People from across the data world came together this week for Strata + Hadoop World 2015 in London. Below we’ve assembled notable keynotes, interviews, and insights from the event.

Shazam already knows the next big hit

“With relative accuracy, we can predict 33 days out what song will go to No. 1 on the Billboard charts in the U.S.,” says Cait O’Riordan, VP of product for music and platforms at Shazam. O’Riordan walks through the data points and trendlines — including the “shape of a pop song” — that give Shazam hints about hits.

Read more…

Comment

Cottage industry 2.0

The O'Reilly Solid Podcast: Amanda Peyton of Etsy on the rise of craft.

Our new episode of the Solid Podcast brings us to Etsy, where David and I spoke with Amanda Peyton, a serial entrepreneur and product manager at Etsy, about the company’s role as a macro-community of micro-communities.

The connection between Etsy and Solid might not be obvious at first. Etsy’s specialty is the ultra-analog: handmade crafts that represent a return to an earlier era of artisan design and manufacturing.

But Etsy is emblematic of how Web platforms have transformed the relationship between product creators and product consumers. It offers rapid feedback from the market, quick discovery of new trends, and access to a large and diverse enough customer base that even extraordinarily niche products can be viable.

The result is a community of distributed manufacturers that’s responsive and efficient. For goods that can be made without a large, well-capitalized factory (even some electronics now fall into this category), Etsy may be the future of manufacturing. Read more…

Comment

The unwelcome guest: Why VMs aren’t the solution for next-gen applications

Scale-out applications need scaled-in virtualization.

scale_in_esterno_Mia_Felicita_Bertelli_FlickrData center operating systems are emerging as a first-class category of distributed system software. Hadoop, for example, is evolving from a MapReduce framework into YARN, a generic platform for scale-out applications.

To enable a rich ecosystem of diverse applications to coexist on these platforms, providing adequate isolation is crucial. The isolation mechanism must enforce resource limits, decouple software dependencies among applications and the host, provide security and privacy, confine failures, etc. Containers offer a simple and elegant solution to the problem. However, a question that comes up frequently is: Why not virtual machines (VMs)? After all, these systems face a number of the same challenges that have been solved by virtualization for traditional enterprise applications.

All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections” — David Wheeler

Read more…

Comment