Electronics make up 40 percent of the cost of a vehicle, according to a 2019 Deloitte report — and at the onset of 2021, a semiconductor shortage stopped the automotive manufacturing industry in its tracks as it sought to address a growing appetite among consumers for new cars.
In short, computers have become a big part of car manufacturing, and those computers run on software written by developers whose day-to-day work is, in many ways, similar to development in other industries. But it’s also different, because of its relationship to specific hardware requirements and the importance of safety and dependability.
Jon M. Quigley started working in the auto industry in 1995, writing software for embedded systems — the industry term for computer chips that run simple software programs on everything from cars to washing machines. These chips, or microcontrollers, have processors and storage like personal computers do and work in much the same way, except they specialize in narrow use cases and are usually built for durability.
“The difference is your laptop isn’t going to go to less than minus 40 degrees Fahrenheit, or have to work in a 120-degree vehicle,” Quigley said.
Hardware durability is always a big concern in the auto industry, even for software developers. Aside from large temperature fluctuations, developers have to worry about factors like how jostling while driving down a rough patch of road can affect the reliability of the software. Depending on where the hardware is mounted, its enclosure may also be subjected to regular vibrations, rock and gravel bombardment, and precipitation and salt spray.
“The difference is your laptop isn’t going to go to less than minus 40 degrees Fahrenheit.”
All of this creates more room for variability and failure than developers in other industries usually face. Traditional software “is not bouncing down the road somewhere,” Quigley said. “It’s not gotten 150,000 miles on it, with a connector that’s a bit sketchy and might give you a goofy signal.”
In his experience, it’s getting the hardware part right that is most difficult in the development process. Every change in design of a vehicle, whether it’s a completely new prototype or an addition to an existing model, has the potential to affect some piece of hardware. The affected hardware has to be reconfigured to fit into the vehicle’s new design, and protected enough from the environment so its software can hope to run consistently.
And unlike on other devices, such as laptops and phones, the end-user can’t always troubleshoot by turning the software off and on again.
“If I have a software fail on my desktop, I’ll probably close it down, restart it, or reboot my machine,” Quigley said. “Those aren’t really options that are always available.”
Risk Analysis and Testing Is a Big Deal
As a result, software development in the auto industry has a strong culture of risk analysis. Before hardware selection starts or a single line of code is written, systems engineers brainstorm all the things that can go wrong and how to mitigate them.
“They are thinking about: ‘What happens if the sensor goes bad? What does the rest of the system do?’” Quigley said. “And if the rest of the system’s natural tendency would be to create some kind of hazardous situation, then they will start putting requirements around the hardware and the software to not allow that to happen.”
He said the risk analysis methods the industry uses are similar to how engineers describe operating at the U.S. Department of Energy, which oversees nuclear reactors and nuclear weapons. Systems are separated into discrete sections, and each section is analyzed to determine how it works and the risks involved. Once risks are assessed, engineers design controls around each system to limit harmful behaviors.
The process is called failure mode and effects analysis (FMEA). For software developers, it can apply to units as small as a function — developers not only have to consider what the function is trying to do, but all the ways the function could fail, why it would fail, and, most importantly, how that failure would impact the larger system.
“One of the things we did was take the vehicle out on the track and run it in tight circles to get a lot of slosh in the fuel.”
Developers aren’t only worried about bugs they write into the code themselves, but also factors that could occur seemingly at random, outside of the developer’s control — something as basic as electrical signals not being strong enough could throw a program off.
As a result, software testing in the auto industry can look a whole lot different than software testing elsewhere. Quigley recalled a time when the marketing department at his company didn’t like the way a vehicle’s fuel-gauge pointer moved under certain kinds of acceleration. In order to recreate and fix the problem, engineers took it out for a spin.
“One of the things we did was take the vehicle out on the track and run it in tight circles to get a lot of slosh in the fuel,” he said. “And then we looked at what the pointer did, and found out the level of dampening for the pointer gauge to make it look appropriate.”
Companies also ship vehicles off to the coldest parts of Canada in the winter and the hottest parts of the U.S. during the summer to test their ability to handle a wide range of temperatures.
“Most of that’s hardware, but the hardware influences the software,” Quigley said.
Many Devs Work on Cross-Functional Engineering Teams
Despite these differences, software development teams at auto companies don’t operate significantly differently than in other industries.
“They use a lot of the same techniques,” Quigley said. “You build an iteration and you test it, you learn something from that iteration, and you can fix those things and add new things to it as you build — that’s the same way we do it in the automotive industry.”
Team structures are different from company to company. Whereas years ago it was more prevalent to have hardware and software teams in separate silos, he said many companies now have groups of “embedded” engineering teams with both hardware and software developers. As a vehicle progresses through different stages of development, teams reshuffle to meet the demands of each stage.
The technology used in the industry is also pretty standard, although tech stacks that get chosen tend to be closer to the hardware, such as assembly in the past, and languages like C more recently.
“Nowadays, memory is so cheap and the integrated circuit dies are so small, nobody’s writing in assembly — or very few.”
“If you had enough resources in your hardware design, then you could choose to use something with a little more overhead,” Quigley said. “Nowadays, memory is so cheap and the integrated circuit dies are so small, nobody’s writing in assembly — or very few.”
One process difference he notes is that the no-estimates trend probably wouldn’t fly in the auto industry. While he sympathized with the idea that it’s hard to gauge timeframes at the beginning of a project when there’s a lot of unknowns, developers in the industry don’t have the luxury of letting the development process dictate the timeline.
“If your margins on a vehicle are very small — and unless you’re a high-end vehicle manufacturer, they probably aren’t very high — you can’t really have a project go over budget every time by a lot,” Quigley said. “You may not know enough to be able to give a good estimate, but in the automotive world you still have to do that. And you’ll have to make the case why your first estimate was not as accurate.”
Car Software Is Only Becoming More Important
Software is becoming a more prominent component of modern vehicles. Earlier this month, Honda began selling the first-ever vehicle to earn the Society of Automotive Engineers’ (SAE) level 3 driving automation designation, on a scale from 0 to 5. The Honda Legend Hybrid EX’s capabilities qualifies as “conditional automation,” and includes the ability to drive autonomously in specific conditions, such as during traffic jams.
Advances in autonomous driving introduces many more variables, and makes software testing much more complex. Virtual testing software is available to help run simulations of tests, but Quigley said it doesn’t solve the issue.
“There’s so many variables involved in how a vehicle actually works in the real world: road surface type, tire type, tire wear, whatever is on the road surface — dirt, water, glare, sensor variation,” he said. “If you think about the number of tests you conduct on a product, it depends on the number of things you’re going to test and the range of those things. So to be able to test an autonomous vehicle fully — it’s an astronomical number of test cases. And simulation only gets you part of the way there.”
At the same time, the amount of software in vehicles raises security and safety concerns. Quigley said that, a few years ago, a researcher demonstrated that it was possible to hack into a car and manipulate vital systems such as the brakes. After that, companies scrambled to develop tighter security and put teams in place to guard against hacks.
“To be able to test an autonomous vehicle fully — it’s an astronomical number of test cases. And simulation only gets you part of the way there.”
“You have Bluetooth engaged on it,” he said, which gives potential attackers entry points into vehicles. The system is open to vulnerabilities because “it’s just standing out there on its own, it’s not behind a firewall or anything.”
As software becomes a larger part of automotive production, more developers will continue to join the industry to find solutions to these challenges. Although it’s common for those in the industry to have more knowledge of hardware than an average developer, Quigley said developers don’t need any existing automotive skills to succeed in the industry. Like most industries, business knowledge is learned on the job. The most important factor is to have a good attitude about continuous learning.
“No matter what trade you’re in, no matter what industry, you’re going to need to have some of that,” Quigley said. “Humility to know that you don’t know everything, and that no matter where you sit, you can learn more. And curiosity to do that too.”