The rise of “apps for everything”

The enterprise IT phenomenon known as systems of engagement can empower developers in varied sectors.

Working on software development tools at IBM, I typically work with enterprise customers. These are large teams building large, complex software systems with a lot of legacy. I was excited when I had a chance to get involved with IBM’s platform-as-a-service (PaaS) initiative, which is now available as BlueMix.

The “systems of engagement” concept behind BlueMix sounded promising, but also triggered my buzzword-skepticism reflex. Ultimately, though, I’ve become convinced there is something happening here, and it extends beyond my traditional enterprise stomping grounds. It’s a projection of a trend that is occurring among makers, enterprise software development shops, and, most surprisingly, in complex systems.

The IT phenomenon “systems of engagement” is actually much bigger than that — it’s a feature of a new era of apps and composable services in software and systems, and it could even be the beginning of a fun phase of software development.

The idea of systems of engagement was put forth by Geoffrey Moore to characterize the mobile, social, and agile faces that businesses are putting in front of their existing business data systems, labeled “systems of record” in the image below. This model resonated with me immediately. I recognized the need for speed in delivering mobile apps — especially event-oriented apps that would have a limited lifespan — and it was also clear there was a need to un-hitch development of these apps from the methodical and conservative rate of development of existing enterprise systems.


Image courtesy of IBM.

Systems of engagement are often built using new PaaS environments. My team started learning these environments in order to build tools for them. We happened to already be making the transition from traditional software delivery to software-as-a-service (SaaS) delivery for our own products, so we had already broken some of the constraints of the old world. We now had one deployment platform, one operating system, one app server, and one database to worry about, and it was up to us to update the software as often as necessary. We were able to move much faster, and we started to think about our existing packaged software as the system of record, providing services to our new fast-moving system of engagement.

I decided to write an article for developerWorks to demonstrate some of the key technologies that are being used in the systems of engagement space. I built a simple Node.js application, using a couple modules that provided access to the Twitter API and a simple sentiment analysis application. Building on top of these reusable modules, I only had to write a handful of Node.js code to stitch together a Twitter sentiment analysis app.

Composition is really starting to work in this new world, in ways we’ve never quite achieved before. For example, a module like ntwitter encapsulates all of the experience interfacing with Twitter. You could say Twitter has become the system of record for social. OAuth2 has matured to where it’s trivial to use, and can now be considered an implementation detail of the Twitter interface module. Paired with a sentiment module that encapsulates a research algorithm for simple dictionary-based sentiment analysis, I cobbled together something that would have previously been inconceivable for an individual to build in an afternoon.

Why is it that modular reuse seems to work so well all of a sudden? I think it’s due to three changes in the developer’s mindset that are occurring today:

  1. Rather than being driven to reinvent the wheel by the traditional “not-invented-here” mindset, the average programmer in communities like Node.js or Ruby is comfortable turning to the Lazyweb for building blocks for software. These programmers are more pragmatic and more focused on the idea than getting all the credit for the software stack on which it’s built.
  2. In the same way that dozens of purpose-built apps on your smartphone have replaced a handful of do-it-all apps on your desktop, modules are built to solve a very specific purpose, and to do it well enough while still offering an extremely simple API. Many modules can be used together to solve a new, more complex problem. The combination of trends no. 1 and no. 2 means a few minutes of searching almost always produces candidate modules to solve a problem, documented by articles or sample apps. And the dynamic nature of Node.js makes it trivial to try them out and select one. This is classic “learning by doing” in the maker spirit.
  3. The industry has rallied around simple APIs using JSON over HTTP as the way to expose capabilities, making those capabilities available on all kinds of client platforms, and making it simple to provide convenience modules to bootstrap applications using them.

It’s obvious these techniques can be applied to systems of record. Enterprise back ends contain the core data that makes a business work; there’s clearly tremendous value locked up in there. Unfortunately, it’s mostly behind traditional technology stacks and APIs. The APIs will need to be dumbed down for consumption in an app, but the potential is clear. You might not immediately allow these new APIs to modify critical business data, but allowing apps to build on things like customer data and purchase history enables a whole range of engaging applications. Teams can take those APIs and experiment with building small, focused apps. These apps, and even the teams building them, probably will have a relatively short life-span. It’s OK to do “quick and dirty” to some degree with these because the value is in time to market and in being able to do many experiments.

The availability of PaaS runtimes like Amazon’s Elastic Beanstalk, Microsoft Azure, Heroku, Cloud Foundry and IBM’s BlueMix also has significant implications. As a developer getting started, I don’t have to configure an environment for a new language or framework I want to experiment with. I don’t have to download, install, and configure the middleware stack I want to use. The promise of the cloud is really working for me; I can literally show up with my code and run. Having built a successful app, I can rely on the platform for scaling and management.

On a lark, I began to share this story with some of our systems customers, working in telecommunications and automotive, for example. It wasn’t clear to me whether it would translate to their domains, but it felt like something I should share. To my surprise, they had exactly the same reaction as my traditional customers: they immediately mapped the systems-of-engagement model to their systems world. The systems of record were the existing components that operated the telecom network or the automotive electronics.

These components have been implemented in a variety of technologies optimized for performance, safety, and compliance with complex requirements and standards, but they also generate a ton of data that could enable countless user-facing applications. The volume of streaming data generated by sensors in the telecommunications and automotive worlds cried out for the same kinds of opportunities the cloud offered to app builders in the IT world. As cars are networked, they can mix local systems-of-record data in the vehicle with services and data in the cloud, just like in the enterprise.

It turns out, IBM and Continental had already begun collaboration in the automotive space, in parallel with IBM’s work with Pivotal on Cloud Foundry. Along the way, someone did the pattern-matching and realized they were building the same architecture, and the connected vehicle project is now building on BlueMix. Rather than building an entire cloud architecture and infrastructure for enabling the construction of apps using vehicle data and streaming vehicle events, they could cut right to the chase and begin prototyping those apps using Node.js on BlueMix using scalable message queuing, time-series data, and data analytics.

One demo we’ve already seen is crowdsourcing weather data by monitoring windshield-wiper usage in traffic. Other automotive suppliers have told me they’re equally as excited at the prospect of enabling this kind of programming model, using the data latent in their components. Much like with enterprise, the automotive IT architecture has a layer that is closed and secure, but there’s an opportunity for a new open, engaging layer built on top.

Along the way, there’s been a second proof point for the concept of apps for vehicles: Apple’s CarPlay:


Apple CarPlay homescreen.

As you can see in the photo above, they’re literally putting apps in the dashboards of cars. It’s not clear yet how open this world will be, but it’s clearly an endorsement of the concepts. Hopefully Google’s Open Auto Alliance will keep them on their toes.

I’ve come to the conclusion that systems of engagement is “apps for everything” — it’s about giving individuals and small teams the tools to take their idea, build on the available data and services, and do something great. The concept was first called out in the context of enterprise IT, but the trend applies to everything from hobbyists to automotive manufacturers and suppliers. It’s truly an exciting time to be a developer.

This post was written by an O’Reilly partner and edited by O’Reilly staff. See our statement of editorial independence.