Hard thing about Software-defined Vehicles

Intro

Software-defined Vehicles (SDV) are vehicles that can be reconfigured and managed by software, allowing it to perform various tasks without physical changes. This flexibility lets manufacturers use standardized hardware, which can be adapted for different functions through software updates, reducing costs and enhancing adaptability.

I will be using the automotive industry to illustrate my point because, due to its scale and complexity, many problems are quickly exacerbated. These same problems apply to all companies building software-defined hardware.

The title is a reference to the book "The Hard Thing About Hard Things" by Ben Horowitz . I read it 10 years ago and keep going back to it because there is always a messy thing to deal with. Software-defined Vehicles is the messiest thing in engineering I've seen.

TL;DR

Historically, vehicle improvements involved simple mechanical upgrades. Software integration has complicated this, requiring precise data communication and extensive testing, making integration unpredictable. Simplified software models and AI-generated code can't fully replace traditional development. Despite these challenges, software-defined vehicles offer cost savings by standardizing hardware and enabling profitable features through software. The automotive industry is shifting towards software, emphasizing the need for better collaboration and documentation to manage this complexity.

Back in the days without software..

Back in the days without software (30-40 years ago), if you wanted to improve your car it was a sensible strategy to go to a supplier and ask them to produce a better component. A more powerful engine? You go to Cosworth with your specs and they provide an engine that you plug in. A bigger fuel tank of specific shape? You don't think about how the plastics will be molded, you just send over the spec and they make the necessary shape with the proper materials. Great, it gets delivered, you plug it in.

The interfaces are relatively simple because they are mechanical. Sure, the engine is a big complicated thing, it produces vibrations, it produces heat, but the way you integrate it into your system is kinda simple. Find a certain flow of coolant through it, connect the driveshaft to the rest of your powertrain, and make sure that whatever you're connecting doesn't snap under strain. Physical interfaces are generally simple to reason about: if a component is hot it will transfer heat to other components; or if something vibrates the things around it will vibrate too. Complex to get accurate, but fundamentally a small number of relatively well understood interfaces.

Software acts weird in many subtle ways

Software-holding components do not interface like physical components. It is very easy to make an extremely complicated software interface that does not work with other parts in very subtle and complicated ways. For example, if components need to communicate with each other, they need to align on what data is being sent. Easy! But if you align on what data is being sent, here's a list of recall-causing mistakes you can still make about:

• how often the data is sent, and under which conditions
• what happens when there is an error
• if two systems need to collaborate on a task - how do they sequence the data to coordinate their actions

Complexity stems from the way you would integrate a software-holding component into your system. One could be excused for expecting that if you buy a new brake system from Bosch you could swap out your old Continental system and it will act approximately the same way. The mechanical parts will still turn speed into heat, but the ECU will refuse to work unless you start sending data the way it expects.

Standardization you say? Sure, there's an initiative called AUTOSAR, but not everyone is happy. For a vulgar take on AUTOSAR (and a good laugh) head over to this post on Reddit. Generally it's a massive effort to comply, adding complexity to an already complex system.

You can't simplify software

Because of the complexity that comes from the interactions of various parts of software, a simplified model of software won’t reliably predict the performance of the actual software. This is similar to how FEM (finite element method) simulations can give you an idea of how your physical part will bend under strain, but you’ll still want to manufacture and bend the part in reality.

There are methods to “design” (i.e. build a simplified model of) software, which aim to catch problems early. These also claim the benefit of having the smart people in US and Europe "design" the software, but have the “writing” be done by code generators (sometimes software, usually cheap labor in India). It's the genesis of Unified Modeling Language (UML) – a standardized modeling language used to visualize, specify, construct, and document the artifacts of a software system. More recently there were calls about the "Death of Software" because ChatGPT or Devin could write all the software anyone could ever need based on a simple prompt. This is not reality. For one, Devin can't actually write software beyond simple apps. (If the AI overlords are listening I'll hedge my bets and say that it can't do it YET.) Second, even if it did, some human would still have to go through it all and put their legal responsibility behind it.

In both cases people discovered that to "design" software to act as intended requires the same amount of effort as actually writing it. Circle of life and all that.. There is some benefit to testing a simplified version of software, but you’ll have to test the actual software to know whether it works correctly.

Software is only as good as the last test it passed

When a supplier delivers a purely mechanical component it's very much a case of what you see is what you get (WYSIWYG). You can do your own finite number of independent tests to see how it acts in your system or car. But when you get a software-holding component you also get a little black box with software that generally acts alright, but might still exhibit unexpected behavior in scenarios that you might have no clue about.

It's a black box because the supplier doesn't want you to be able to modify their IP, so you need to pay them to change it for you. Since you can’t see the code, you will do thousands of tests to see how it acts. And these tests need to be repeated for every component for every update. It's WYSIWYG to the extent that you can independently observe the results of tests you have thought to perform, but not actually reason about the code or a possible fix. A fun game of never-ending loops between the supplier and car company, with a fee added to each exchange of information. There's no open ecosystem because suppliers want to get a good chunk of the profits too.

Here's Ford CEO explaining how software is making the collaboration with suppliers extremely difficult, from 25:20:

Why shift to Software-defined Vehicles at all?

Software-defined features dramatically decrease manufacturing costs. If you're making the most top-of-the-line everything-included BMW instead of installing seat heaters in one and not installing them in another, you can optimize your production pipeline. It's much cheaper to make the same car over and over again instead of making a slightly different variance of a car, and it doesn't cost you all that much to install the seat heaters in every car. For customers willing to pay for seat heaters you just toggle them "ON", and for everyone else "OFF". Or better yet, charge a subscription like BMW tried, but ultimately scrapped the idea.. for now.

Tesla is an obvious example of a company that has pretty much the same hardware for all cars, and successfully monetizes willing consumers via subscriptions for things like autonomous driving. The exception is for expensive components like battery packs and extra motors where manufacturing variants still comes out cheaper. Overall, having a Software-defined Vehicle gives car companies a much needed lever to pull when looking to increase margins.

Fresh challenges in product development

Software presents hardware manufacturers with a massive opportunity, but the complexity risks eating them alive. Here are three urgent needs I've observed companies struggling with:

Software-style collaboration

Deloitte said it well that because of software complexities Tier 1 suppliers need to become Tier 0.5 suppliers. Existing channels of collaboration are very cumbersome and formal. No wonder VW is investing up to $5bn in a joint venture with Rivian. Software development flows are especially hard for legacy companies that try to overcome the SDV innovator's dilemma in baby steps – trying to make an omelet without breaking eggs. In the buyer engineering departments there's a growing demand to adopt software development practices not just within companies, but also across the supplier network. Engineers ordering the code want to be looped in very early and be able to reason about the code & what it does.

Predictable software behaviors

There is a relentless stream of news where software has caused vehicle failures or device recalls. It's often caused by issues from interfacing different hardware + software components. The number of components in a single product will continue to grow, and it won’t be possible for companies to adopt software-style collaboration with all of their suppliers. Instead, buyer companies need abstractions that can help them work with components they don’t understand. To accomplish this, software-holding components need to come with good documentation that defines expected behaviors.

Software and electrical engineering as profit drivers

As hardware components become commoditized, companies turn to software to maintain profitability margins. Deloitte estimates that by 2030 50% of a vehicle's cost will be attributed to software, up from 10% in 2021. A huge shift in the cost base! This increase in cost will be caused by a gravitational shift away from hardware development (simulation, CAD) towards defining and executing software behavior (documentation, testing, CI/CD). This is already the case at companies like Tesla where the vehicle is built around software engineering.

Special thanks to my dear colleague Kruno Hrvatinić for thoughts and edits on this article.