"software" entries

Designing resilient communities

Establishing an effective organization for large-scale growth

In the open source and free software movement, we always exalt community, and say the people coding and supporting the software are more valuable than the software itself. Few communities have planned and philosophized as much about community-building as ZeroMQ. In the following posting, Pieter Hintjens quotes from his book ZeroMQ, talking about how he designed the community that works on this messaging library.

How to Make Really Large Architectures (excerpted from ZeroMQ by Pieter Hintjens)

There are, it has been said (at least by people reading this sentence out loud), two ways to make really large-scale software. Option One is to throw massive amounts of money and problems at empires of smart people, and hope that what emerges is not yet another career killer. If you’re very lucky and are building on lots of experience, have kept your teams solid, and are not aiming for technical brilliance, and are furthermore incredibly lucky, it works.

But gambling with hundreds of millions of others’ money isn’t for everyone. For the rest of us who want to build large-scale software, there’s Option Two, which is open source, and more specifically, free software. If you’re asking how the choice of software license is relevant to the scale of the software you build, that’s the right question.

The brilliant and visionary Eben Moglen once said, roughly, that a free software license is the contract on which a community builds. When I heard this, about ten years ago, the idea came to me—Can we deliberately grow free software communities?

Read more…

Comment

Code Simplicity: The science of software design

Learn to be a better programmer by taking charge of your interests

If you want to be a better programmer, a good first step would be to choose an area of software development to take additional responsibility for. Now, when we say “responsibility,” we don’t mean the sort of “you’re to blame and you accept it” responsibility that is so commonly thought of. Instead, we mean the willingness to take charge or the willingness to do something about an area.

So select out some area of software development, and decide to be a bit more responsible for it than one was before. The “area” could be simply some additional part of the current project you work on, the whole project itself, some type of software development that you haven’t done before, some aspect of software development you’d like to know more about, etc. If you’re feeling adventurous, try deciding that you’re personally responsible for the quality of the entire software project you’re working on. If you do, you may be surprised at how much easier this makes your life. When you’re trying to maintain the quality of only a piece of a software project, it’s very difficult. You’re surrounded by complexity or confusion, and you have to fend it off at every turn. But when you look at the project as a whole instead and try to decide what should be done with it as a whole, the solution presents itself much more readily. Now, it may seem like quite a bit more work to resolve the problems of the whole project, and it is. But it’s considerably more satisfying, tremendously more effective, and will bring you up to seniority as a software developer much more quickly than just trying to solve the problems of your one particular area.

Read more…

Comments: 2

New resource for developers

O'Reilly's new site for all things related to programming.

Welcome to O’Reilly Media’s Programming blog, our resource for all things related to programming. Whether you’re a professional developer, hardcore hacker, or student, I hope this site provides you with interesting ideas, ways to learn new skills, exposure to alpha geeks, and the opportunity to interact with our talented and unique editors. The best part of my days are conversations with our editors and authors, and I’d like you to benefit from the same exchange of ideas.

We’re building this blog to meet your needs with news, information, and analysis. Whether you work on front ends, back ends, or middleware, open source software or commercial, there’s something here for you. Over the coming weeks and months we’ll be publishing how-to information, interviews, and our opinions in addition to exposing you to O’Reilly’s vast products and services.

We’ve seen an explosion of interest in the creation of software over the last two years. Groups like Codecademy promise to teach anyone willing to code, even people like Mayor Bloomberg of New York; one of the most important factors in the legal fight between Google and Oracle was a judge who knew how to code; and groups like Code for America are transforming the civic landscape.

With O’Reilly’s Programming blog, we intend to serve a broad and diverse group. If you’re a web designer, a C# developer working in SharePoint, creating more efficient JavaScript code, need to figure out how to build apps for mobile and other devices, a designer building effective user experiences, optimizing your company’s websites, deciding whether to deploy an app in the cloud, think Perl still rocks, or trying to get your product development group to work better together, we’ll provide you with the tools and information to be productive.

Open source remains one of the pillars of the programming community, but we’re building a very large tent — a tent that includes Windows developers, iOS developers, Oracle developers, and more. We’ll also pay attention to non-technical issues that affect programmers: jobs, developer culture, and occasional tangents into other obsessions (like food).

O’Reilly’s readers created the world we inhabit. Now, we need to bring a new generation into that world and expand it for those already making a difference. While we have a lot of ideas for the Programming blog, we want to hear your thoughts about what you want to see on the site and how you want to participate. Please let us know what you think through the comments or email me directly.

Comment

The coming of the industrial internet

Our new research report outlines our vision for the coming-together of software and big machines.

The big machines that define modern life — cars, airplanes, furnaces, and so forth — have become exquisitely efficient, safe, and responsive over the last century through constant mechanical refinement. But mechanical refinement has its limits, and there are enormous improvements to be wrung out of the way that big machines are operated: an efficient furnace is still wasteful if it heats a building that no one is using; a safe car is still dangerous in the hands of a bad driver.

It is this challenge that the industrial internet promises to address by layering smart software on top of machines. The last few years have seen enormous advances in software and computing that can handle gushing streams of data and build nuanced models of complex systems. These have been used effectively in advertising and web commerce, where data is easy to gather and control is easy to exert, and marketers have rejoiced.

Thanks to widespread sensors, pervasive networks, and standardized interfaces, similar software can interact with the physical world — harvesting data, analyzing it in context, and making adjustments in real-time. The same data-driven approach that gives us dynamic pricing on Amazon and customized recommendations on Foursquare has already started to make wind turbines more efficient and thermostats more responsive. It may soon obviate humans as drivers and help blast furnaces anticipate changes in electricity prices. Read more…

Comments: 7

Let’s do this the hard way

Being both liberal and safe in programming is hard

Recent discoveries of security vulnerabilities in Rails and MongoDB led me to thinking about how people get to write software.

In engineering, you don’t get to build a structure people can walk into without years of study. In software, we often write what the heck we want and go back to clean up the mess later. It works, but the consequences start to get pretty monumental when you consider the network effects of open source.

cartoon-37304_640You might think it’s a consequence of the tools we use—running fast and loose with scripting languages. I’m not convinced. Unusually among computer science courses, my alma mater taught us programming 101 with Ada. Ada is a language that more or less requires a retinal scan before you can use the compiler. It was a royal pain to get Ada to do anything you wanted: the philosophical inverse of Perl or Ruby. We certainly came up the “hard way.”

I’m not sure that the hard way was any better: a language that protects you from yourself doesn’t teach you much about the problems you can create.

But perhaps we are in need of an inversion of philosophy. Where Internet programming is concerned, everyone is quick to quote Postel’s law: “Be conservative in what you do, be liberal in what you accept from others.”

The fact of it is that being liberal in what you accept is really hard. You basically have two options: look carefully for only the information you need, which I think is the spirit of Postel’s law, or implement something powerful that will take care of many use cases. This latter strategy, though seemingly quicker and more future-proof, is what often leads to bugs and security holes, as unintended applications of powerful parsers manifest themselves.

My conclusion is this: use whatever language makes sense, but be systematically paranoid. Be liberal in what you accept, but conservative about what you believe.

Comment: 1

The software-enabled cars of the near-future (industrial Internet links)

Ford's OpenXC platform opens up real-time drivetrain data.

OpenXC (Ford Motor) — Ford has taken a significant step in turning its cars into platforms for innovative developers. OpenXC goes beyond the Ford Developer Program, which opens up audio and navigation features, and lets developers get their hands on drivetrain and auto-body data via the on-board diagnostic port. Once you’ve built the vehicle interface from open-source parts, you can use outside intelligence — code running on an Android device — to analyze vehicle data.

Of course, as outside software gets closer to the drivetrain, security becomes more important. OpenXC is read-only at the moment, and it promises “proper hardware isolation to ensure you can’t ‘brick’ your $20,000 investment in a car.”

Still, there are plenty of sophisticated data-machine tieups that developers could build with read-only access to the drivetrain: think of apps that help drivers get better fuel economy by changing their acceleration or, eventually, apps that optimize battery cycles in electric vehicles.

Drivers with Full Hands Get a Backup: The Car (New York Times) — John Markoff takes a look at automatic driver aides — tools like dynamic cruise control and collision-avoidance warnings that represent something of a middle ground between driverless cars and completely manual vehicles. Some features like these have been around for years, many of them using ultrasonic proximity sensors. But some of these are special, and illustrative of an important element of the industrial Internet: they rely on computer vision like Google’s driverless car. Software is taking over some kinds of machine intelligence that had previously resided in specialized hardware, and it’s creating new kinds of intelligence that hadn’t existed in cars at all. Read more…

Comment: 1

The future of programming

Unraveling what programming will need for the next 10 years.

Programming is changing. The PC era is coming to an end, and software developers now work with an explosion of devices, job functions, and problems that need different approaches from the single machine era. In our age of exploding data, the ability to do some kind of programming is increasingly important to every job, and programming is no longer the sole preserve of an engineering priesthood.

Is your next program for one of these? Photo credit: Steve Lodefink/Flickr.

Is your next program for one of these?
Photo credit: Steve Lodefink/Flickr.

Over the course of the next few months, I’m looking to chart the ways in which programming is evolving, and the factors that are affecting it. This article captures a few of those forces, and I welcome comment and collaboration on how you think things are changing.

Where am I headed with this line of inquiry? The goal is to be able to describe the essential skills that programmers need for the coming decade, the places they should focus their learning, and differentiating between short term trends and long term shifts. Read more…

Comments: 34

Interoperating the industrial Internet

If we're going to build useful applications on top of the industrial Internet, we must ensure the components interoperate.

One of the most interesting points made in GE’s “Unleashing the Industrial Internet” event was GE CEO Jeff Immelt’s statement that only 10% of the value of Internet-enabled products is in the connectivity layer; the remaining 90% is in the applications that are built on top of that layer. These applications enable decision support, the optimization of large scale systems (systems “above the level of a single device,” to use Tim O’Reilly’s phrase), and empower consumers.

Given the jet engine that was sitting on stage, it’s worth seeing how far these ideas can be pushed. Optimizing a jet engine is no small deal; Immelt said that the engine gained an extra 5-10% efficiency through software, and that adds up to real money. The next stage is optimizing the entire aircraft; that’s certainly something GE and its business partners are looking into. But we can push even harder: optimize the entire airport (don’t you hate it when you’re stuck on a jet waiting for one of those trucks to push you back from the gate?). Optimize the entire air traffic system across the worldwide network of airports. This is where we’ll find the real gains in productivity and efficiency.

So it’s worth asking about the preconditions for those kinds of gains. It’s not computational power; when you come right down to it, there aren’t that many airports, aren’t that many flights in the air at one time. There are something like 10,000 flights in the air at one time, worldwide; and in these days of big data, and big distributed systems, that’s not a terribly large number. It’s not our ability to write software; there would certainly be some tough problems to solve, but certainly nothing as difficult as, say, searching the entire web and returning results in under a second. Read more…

Comment: 1

To eat or be eaten?

What's interesting isn't software as a thing in itself, but software as a component of some larger system.

One of Marc Andreessen’s many accomplishments was the seminal essay “Why Software is Eating the World.” In it, the creator of Mosaic and Netscape argues for his investment thesis: everything is becoming software. Music and movies led the way, Skype makes the phone company obsolete, and even companies like Fedex and Walmart are all about software: their core competitive advantage isn’t driving trucks or hiring part-time employees, it’s the software they’ve developed for managing their logistics.

I’m not going to argue (much) with Marc, because he’s mostly right. But I’ve also been wondering why, when I look at the software world, I get bored fairly quickly. Yeah, yeah, another language that compiles to the JVM. Yeah, yeah, the Javascript framework of the day. Yeah, yeah, another new component in the Hadoop ecosystem. Seen it. Been there. Done that. In the past 20 years, haven’t we gained more than the ability to use sophisticated JavaScript to display ads based on a real-time prediction of the user’s next purchase?

When I look at what excites me, I see a much bigger world than just software. I’ve already argued that biology is in the process of exploding, and the biological revolution could be even bigger than the computer revolution. I’m increasingly interested in hardware and gadgetry, which I used to ignore almost completely. And we’re following the “Internet of Things” (and in particular, the “Internet of Very Big Things”) very closely. I’m not saying that software is irrelevant or uninteresting. I firmly believe that software will be a component of every (well, almost every) important new technology. But what grabs me these days isn’t software as a thing in itself, but software as a component of some larger system. The software may be what makes it work, but it’s not about the software. Read more…

Comments: 17

Tracking Salesforce’s push toward developers

Salesforce's recent investments suggest it's building a developer-centric suite of tools for the cloud.

SalesforceHave you ever seen Salesforce’s “no software” graphic? It’s the word “software” surrounded by a circle with a red line through it. Here’s a picture of the related (and dancing) “no software” mascot.

Now, if you consider yourself a developer, this is a bit threatening, no? Imagine sitting at a Salesforce event in 2008 in Chicago while Salesforce.com’s CEO, Marc Benioff, swiftly works an entire room of business users into an anti-software frenzy. I was there to learn about Force.com, and I’ll summarize the message I understood four years ago as “Not only can companies benefit from Salesforce.com, they also don’t have to hire developers.”

The message resonated with the audience. Salesforce had been using this approach for a decade: Don’t buy software you have to support, maintain, and hire developers to customize. Use our software-as-a-service (SaaS) instead.  The reality behind Salesforce’s trajectory at the time was that it too needed to provide a platform for custom development.

Salesforce’s dilemma: They needed developers

This “no software” message was enough for the vast majority of the small-to-medium-sized business (SMB) market, but to engage with companies at the largest scale, you need APIs and you need to be able to work with developers. At the time, in 2008, Salesforce was making moves toward the developer community. First there was Apex, then there was Force.com.

In 2008, I evaluated Force.com, and while capable, it didn’t strike me as something that would appeal to most developers outside of existing Salesforce customers.  Salesforce was aiming at the corporate developers building software atop competing stacks like Oracle.  While there were several attempts to sell it as such, it wasn’t a stand-alone product or framework.  In my opinion, no developer would assess Force.com and opt to use it as the next development platform.

This 2008 TechCrunch article announcing the arrival of Salesforce’s Developer-as-a-Service (DaaS) platform serves as a reminder of what Salesforce had in mind. They were still moving forward with an anti-software message for the business while continuing to make moves into the developer space. Salesforce built a capable platform. Looking back at Force.com, it felt more like an even more constrained version of Google App Engine. In other words, capable and scalable, but at the time a bit constraining for the general developer population. Don’t get me wrong: Force.com wasn’t a business failure by any measure; they have an impressive client list even today, but what they didn’t achieve was traction and awareness among the developer community. Read more…

Comment: 1