That thing looks like hardware, but it’s software now

Building great software on time is at the heart of more and more "hardware" projects.

X-35A_JSF

The X-35A JSF performs flight tests at Edwards Air Force Base, California. Photo: Wikimedia Commons.

I saw this piece in the U.S. Naval Institute News today that notes software delays could translate into less effective Joint Strike Fighters. (It’s based on the GAO’s report that can be found here.) I also read somewhere that the biggest difference between versions used by our domestic armed forces and those sold overseas would be their software load.

I think what’s most interesting about this story is that when we look at an airplane we tend to see a physical thing. We see airfoils, materials, and hard sciences animated and airborne, but a growing proportion of the “thingness” of these machines is happening in software — software that makes it fly and software that connects it with all the other things on the battlefield, to share information and fight as one organism.

This airplane will require approximately eight million lines of code on board to run mission systems and flight sciences. I’m guessing flight sciences code will be the same for the U.S. and its partner / buyers, but I’m not sure. Given that the aircraft is flying but not operational, one could hazard a guess that the flight sciences code is coming along faster than the mission stuff with all its complex real-time target fusion stuff going on. And the mission code is the really interesting part. It’s what makes a single aircraft part of a bigger whole. It’s analogous to what makes the Nest more than just your typical thermostat, but much, much more.

I don’t know what proportion of the JSF weapon system cost is in software, but in addition to the eight million lines of code in the aircraft, another 16 million lines are expected for off-airframe systems. Everything from logistical pipelines to command and control systems either have to be written or updated for this aircraft to seamlessly fit into the rest of the force. Given that this code is written and tested under some of the most conservative rules, my guess is that coding will account for something like a thousand person years of effort in the JSF program. Actually, that number is probably low. All this code is showing up in the aircraft’s development costs and, more importantly, its development risk and timelines.

That’s still not a big cost relative to the total cost of the program, but the risk inherent in timely software delivery certainly is turning out to be big. The airplane is there. It’s flying. But it won’t be ready to face an adversary before 2015 at the very earliest (and this report suggests that timeline is now broken) because they simply can’t get the software written, tested and certified fast enough.

The Air Force hires contractors that started as air frame designers and manufacturers to build things like the JSF. These companies write a lot of software now, but I don’t think anyone would suggest they are at the leading edge of what is happening in the discipline. America’s most expensive weapons system ever is waiting for what used to be an afterthought to the design process before it can become operational.

While this story might seem confined to aerospace and its complex systems, I suspect this is a harbinger for the kinds of challenges a number of industries will face. We’ll probably start to see variations of this dynamic pop up in all kinds of places now that the things we’ve always seen as hardware turn into containers of more and more critical software. When even the humble window air conditioner is shipping with a lot of software inside, the ability to develop good software on short timelines is going to be important to many companies that didn’t previously have any experience with that. The auto industry is certainly beginning to grapple with this. What about the industry you’re in?

tags: , ,

Get the O’Reilly IoT+ Newsletter

Software / Hardware / Everywhere

The programmable world is creating disruptive innovation as profound as the Internet itself. Be among the first to learn about the latest news, trends, and opportunities.

  • deemery

    I don’t know about the assertion
    “That’s [software cost] still not a big cost relative to the total cost of the program,”

    Software, and particularly software verification, is becoming an increasing cost -driver- for both commercial and military aircraft. Software test delays are being blamed for F35 schedule slip, and were blamed for at least part of the Washington Metro “Silver Line” Phase I schedule slip.

    And I’m bothered by this assertion, too:
    “The Air Force hires contractors that started as air frame designers and manufacturers to build things like the JSF. These companies write a lot of software now, but I don’t think anyone would suggest they are at the leading edge of what is happening in the discipline. ”
    Writing software for safety critical systems is NOT like writing web applications. Trendy things like “agile” and “scrum” won’t scale to meet complexity, real-time behavior, fault analysis/fault mitigation and verification regimes such as DO-178C.

    Frankly, I submit the author of this piece might want to do some more research about this domain.

    • Jim S

      Good point. I just don’t know the split in costs. I simply estimated the cost of 1000 person years (which itself is a guess) and compared that to total program costs. Relative to that total it’s low. Relative to just development costs it is probably substantial.

    • deemery

      Here is a 4 line introduction into large DoD program funding: There are 3 “colors of money”, RDT&E, OPA and OMA. RDT&E money pays for the R&D through production of prototypes, and that includes software development and verification. The software costs are a huge part of RDT&E. OPA money buys the production runs, i.e. actually pays for aircraft delivery, and OMA provides the support for the aircraft.

      • Jim S

        Thanks. I did DoD work so I have basic familiarity with that. What I don’t know (and didn’t try to find out – my bad) was the actual split in RDT&E between airframe development and software. Do you know where those figures might be available?

  • keegan

    I think the following assertion is totally fair:

    “The Air Force hires contractors that started as air frame designers and manufacturers to build things like the JSF. These companies write a lot of software now, but I don’t think anyone would suggest they are at the leading edge of what is happening in the discipline. ”

    Maybe “agile” or “Scrum” “won’t scale” (I don’t think that is true), but that doesn’t make the assertion false. The aerospace industry is behind the curve. The author’s assertion (or maybe more accurately, observation) is correct, although lacking any knowledge of the process for certifying aircraft systems the author may not understand why the industry is behind, or be able to appreciate the complexity of trying to catch up. As an “insider” I agree with his observation, but having a deeper understanding of the process I can perhaps lend some thoughts as to why I feel this observation is true.

    The objectives of DO-178B and DO-178C can certainly be achieved using agile development methodologies. DO-178B/C is about ensuring certain processes are performed, ensuring certain objectives are met, and that evidence is obtained to demonstrate that you have achieved the required objectives. This does not prevent the use of agile methods in your development. What it does mean though, is that if you choose to use an agile methodology, there needs to be an understanding of how to map that practice to the industry or regulatory standards you may be bound to. That is often easier said than done, and certainly contributes to the industry having a hard time adopting such methodologies.

    deemery’s statement about the cost of software verification is something I’ve observed to be true as well. I think this is an area where the adoption of some more modern practices like Test Driven Development or Behavior Driven Development could go a long way. The trick is learning how to leverage those newer methods in an environment where regulatory and certification requirements need to be met as well.