The first few minutes riding in a driverless vehicle are exhilarating, a little scary and awe-inspiring. It feels surreal, like a scene from a futuristic Blockbuster movie.
But after that?
“My first ride was remarkably boring,” said Justin Wood. “I was beside myself at first as the car started to navigate itself around the city. The steering wheel spinning around with no driver to control it was a curiosity that left me laughing. But after a few minutes, you forget about the steering wheel and settle into utter normalcy.”
After the novelty wears off, riders feel safe, and safety is the top priority at Cruise, where Wood works as an engineering manager of fleet health and has taken plenty of spins in driverless cars.
Contrary to what films want us to believe — The Love Bug, Knight Rider, Total Recall — riding in driverless vehicles shouldn’t be very exciting. The real thrill is behind the scenes. At Cruise, team members are working with captivating technology to make autonomous vehicles universal and benign — and, according to Wood, to deliver upon “the promises of a science fiction future.”
The scale and impact of the work at Cruise is staggering. Cruise has completed more than 3 million miles of autonomous testing, and its fleet is exposed to more than 20,500 intersections, 3,000 cut-ins and 3,000 double-parked cars every day. “We get to work on some of the hardest engineering challenges that exist, solving problems that have never been solved before. It’s the cutting edge of machine learning and research. Someday, road fatalities will be a thing of the past,” said Senior Software Engineer Chris Hasson.
Backing Cruise
Cruise is also a huge company with nearly 3,000 employees — two-thirds of which are engineers. Supporting that scale of development is demanding, and creating innovative, custom internal tooling has presented interesting scaling challenges of its own. That’s where unique solutions such as Cruise Shells, WebViz and Starfleet come in. These behind-the-scenes tools may not get all of the spotlight, but the work at Cruise would not be possible without them.
Built In got in the passenger seat with some of the technologists who built these solutions — Wood, Hasson and Software Engineering Manager Sireesha Sura — for insight into the interesting problems that keep them engaged on the job.
Justin Wood spends his days working on Starfleet — the in-house tool to give the operations team visibility into each Cruise autonomous vehicle on the streets of San Francisco. It started as a source of truth to track the vehicles, but recently Cruise has started transitioning away from reliance on safety drivers to fully driverless vehicles. That comes with edge-case operational concerns — outside of driving — such as monitoring the battery. Wood and his colleagues are using Starfleet to build a system to inform these scenarios and facilitate human intervention through alerts or Slack messages.
“Using this interface, users can see all sorts of operational data about the status of the vehicles, including their allocation, the route they are currently executing, the time to the next waypoint and all kinds of configuration data. At a glance, operators can see if AVs are experiencing a delay and make decisions about how to react to such situations,” said Wood.
The tool needs to scale to keep up with the upcoming expansion: Cruise will be increasing the size of its ridehail fleet exponentially and entering new geographies, including Phoenix and Austin.
Here’s what Wood had to say:
About riding in autonomous vehicles: “All of this innovation and commitment is leading us to a future where I fully believe autonomous vehicles will be a central fixture in how we get from place to place. They are cleaner, safer, more affordable and without the burden of having to do the work of driving yourself. It is truly a brighter future we are working towards.”
About the culture of innovation: “Having joined Cruise fairly recently, one thing I find myself telling people is how impressed I am with the pace of innovation here. It isn’t limited to the technology, either; innovation seems to be core to the very essence of what it means to be a Cruiser, regardless of what function you serve or what team you work on. I am honored to work with a group of people to whom ten-fold performance improvements are just something they do on an average Wednesday, or who brainstorm novel approaches to intractable problems for fun.”
“I am honored to work with a group of people to whom ten-fold performance improvements are just something they do on an average Wednesday.”
About the scale of the work: “The Fleet team is full of amazing talent and these past few quarters have seen us working on enormous initiatives, beyond the scope of even our own team’s formidable capabilities. However, leadership saw the value of the undertaking and revealed some innovation of their own, committing additional development resources from across the broader organization to ensure we are able to deliver on the critical initiatives that will allow Starfleet and Jumpstart to scale to tens of thousands of autonomous vehicles and meet our company goals. It is inspiring to see an entire organization rally around a goal.”
About the impact: “Fleet team’s flagship product, Starfleet, is not just one tool, but rather a whole suite of products that have a strong impact on Cruise’s day-to-day operations, functioning at a broader scale than most. For example, while Remote Assistance, another very important tool, performs its functions against a single AV, Starfleet generally executes its functions across multiple AVs at a time, including against a full fleet simultaneously. This allows Cruise to respond efficiently to events that affect large geographical areas, like instructing a fleet to avoid bad weather, or that require coordination across many vehicles, such as summoning a group of AVs back to a facility after a shift. Without a tool that can work at such a scale, we could never hope to achieve our scaling ambitions.”
Cruise Shells is an in-house tool that allocates developers’ customized, ephemeral pre-configured virtual machines through the Google Cloud Platform each day. It’s like a buffet for engineers, who can order features a la cart in accordance with their dev environment needs each day. “Our hardware team tests on multiple versions of hardware, and our AI team likes running models on different systems. So, instead of asking IT to procure new hardware for you, you can send a flag to Cruise Shell that says, ‘Today I want a GPU instead of a CPU,’” said Sireesha Sura, whose team of eight manages the tool.
The tool saves Cruise money, resources and loads of time — what might otherwise be a four-hour update can be done instantaneously. Then, when developers are done each evening, Cruise kills the VM and the developers can start with a new custom environment the next day.
Here’s what Sura had to say:
About riding in autonomous vehicles: “I don’t like making risky decisions. It’s very scary sitting in the car for the first time, you don’t know what it’s going to do. Cruise has put so much effort into making sure it’s as safe as it can be.”
About the culture of innovation: “We have hackathons at the end of each quarter. People take three days to work on ideas. My entire team was the outcome of a hackathon idea at the end of 2019. We formed a tiger team and started out with beta customers, and it really stuck. So we made a team out of it, and now we have eight engineers.”
About data: “Data plays a huge role in deciding what the team will work on. For example, our repository size has been growing over time, so people have been asking for more disk space so they can clone a repo and work with it. If you have two clones of our repository, that’s more than a terabyte of disk space. That space is not cheap. What we can do in the dev prod org is either give people more disk space or reduce the amount of space that the repo is taking.”
“We have hackathons at the end of each quarter. My entire team was the outcome of a hackathon idea at the end of 2019.”
About scaling: “We have about 1,000 engineers using Cruise Shells. Over the summer, we had an intern work on a project that saves about 2 minutes pulling up a Cruise Shell instance every morning. That doesn’t sound like much time, but when you compute that over 500-1,000 engineers using it every day for a year, it ends up saving an entire engineer’s worth of time. This is increasingly important as Cruise hires smarter and smarter people who want to do multiple tasks simultaneously — one Cruise Shells instance isn’t sufficient for them, they want five in parallel, with five different configurations.”
In his work on WebViz, Hasson is in a constant race against data. WebViz is a browser-based tool that gives employees across teams a view of the data captured by the car through various visualizations.
It runs like a video. “It even has a future time slider, where you can see what the car predicts will happen in the next one, three, five or ten seconds,” said Hasson. “You can pull up images from its cameras, or plots and graphs of the car’s velocity.”
Hasson says the tool “pushes browsers to the limits of their capabilities.”
Here’s what else Hasson had to say:
About riding in autonomous vehicles: “We can take friends on rides now. My friend was terrified for the first couple of minutes, and then by the end of the ride he was asleep. That’s how good ‘boring’ is when it comes to autonomous vehicles.”
About data: “The car captures a huge amount of data at any given moment. One of the challenges for WebViz is ensuring we pull the most important subset to ensure it loads quickly. If an engineer is curious what happened in a 10- second interval of driving, it can be on the order of gigabytes worth of data. At the end of the day, this is all just a tab in Chrome. So, what WebViz has spent a lot of time trying to figure out is the minimum set of data we can load in order to give engineers a quick sense of what happened.”
“The car captures a huge amount of data at any given moment. WebViz pulls the most important subset and ensures it loads quickly.”
About the culture of innovation: “There are so many different teams that use WebViz that we don’t have enough developers to build all of the features everyone is interested in. So we built the WebViz Playground, which is a code editor in the website that allows people to modify the code as they need instead of asking WebViz engineers. It lets teams solve their own problems, and it’s very customizable. It was also the product of a hackathon in fall of 2017.”
About the best part of the job: “Of all the websites I’ve been able to work on, WebViz really pushes what’s possible of a web browser to the absolute limit. We use all of the coolest tools, like WebGL, Web Assembly, Web Workers. We’re doing things that people didn’t even think were possible.”
