Volvo Leverages AI and Virtual Worlds to Revolutionize Car Safety

Volvo reconstructs crashes with AI in virtual worlds to make safer cars

Safety is Volvo's big thing. From three-point seatbelts to side-impact protection to blind-spot monitoring and much more, the Swedish automaker focuses its innovation on making its cars safer rather than faster around the Nurburgring. (Although in the past, it's made them faster, too -- sort of.) As part of its efforts, Volvo has collected data from tens of thousands of car crashes, which it's now leveraging in virtual worlds to put simulated cars and SUVs through the wringer with the help of something amusingly named "Gaussian splatting." "We've been visiting crash sites since the '70s. We've been recording event data from from a fleet of cars for many, many years," said Alwin Bakkenes, head of global software engineering at Volvo Cars. "And all of those data points have actually helped us create safety innovations... from the three-point safety belt to whiplash protection systems and now also, as we're introducing in the ES90, a function called 'lidar AS,' which helps the car steer away from vulnerable road users in the dark," he said. Like some others in the industry, Volvo is an early adopter of the software-defined vehicle, or SDV. As a quick refresher, SDVs are clean-sheet designs in terms of their electronic architecture. Instead of dozens or even a hundred discrete black boxes, each with its own hardware and running its own software, each doing a discrete job (like controlling the air conditioner or managing traction control), four or five powerful central computers take over those roles, overseeing domains like infotainment, advanced driver assistance systems, handling and powertrain, and interior comfort. "One thing is that we develop this in-house now, so instead of relying on suppliers with long deadlines and long process and sending requirements back and forth, we develop the software," said Erik Coelingh, VP of product at Zenseact, an ADAS developer owned by Volvo. "If there's something, we solve it in a day," Coelingh said. "It's so much faster. So we iterate much faster. As Alwin said, we're all testing with the new software every single day. So the innovation speed is fundamentally different than before, and the way we try to use this is to really build safety and go toward lower accident rates in a pace that we've never seen before." Among the advantages of moving to SDVs is that it's much easier to simulate them since the entire software stack can be run virtually. That's why Volvo has built one of the largest data centers in Europe: to be able to run those sims. Like a lot of other companies out there, Volvo has turned to AI to speed up the development process. But how does such a safety-conscious company like Volvo know it can trust the output of those end-to-end algorithms? "Gaussian splitting is a technology where we can take one point, one traffic scenario, and explode it into thousands or tens of thousands of scenarios from this real-world data," Coelingh said. "And then we can manipulate one scenario into a thousand different scenarios, and then we can enclose the simulation and test our software against this." Autonomous vehicle developers have been simulating in environments like Unreal Engine for some time now. "That's very visual; that works for camera data. But here we're probing lidar data, camera data, radar data, and we reconstruct the scene with the neural net, and then do the manipulation and use closed-loop simulation," Coelingh said. "So this is a way of really fast be able to test your software against a huge, huge amount of different scenarios that are representative for the real world." The first neural radiance fields were able to interpolate 3D objects from an input of 2D images taken from different angles. "So instead of doing all the physics calculations on how the rays bounce and all this, instead, you train a neural network to learn the static scene," Coelingh said. "The next step was to build NeRFs in 4D -- in space time. So we are using this for, let's say, a camera sequence of 10 seconds, and then you can, in time, reconstruct this." Running the original data with the original conditions in simulation lets Volvo check that the simulation is faithful enough to recreate the original outcome. From there, it can start changing conditions, rerunning the test to see how (or if) the car's behavior changes. By doing the same for radar and lidar data as well as camera data, "we can reconstruct the scenarios that the car sees in reality... with all the strange things that happen in the real world," Coelingh said. "We have our millions and millions of data points where we actually have events that we collect from our fleet," Bakkenes said. "When we find something the system finds difficult to deal with -- a scenario -- then we can zoom into one of those scenarios with all the sensor data, and we can explode that into a thousand variants of that particular scenario." Virtual testing with Gaussian splatting has not replaced real cars driving on the test track, but it does let Volvo test new cars in a much wider range of conditions than would otherwise be possible.

Volvo recreates unusual road incidents with AI for safer ADAS features

This new tech lets Volvo fine-tune how its ADAS features react in an infinite range of on-road scenarios Volvo has taken to a cutting-edge technique for reconstructing scenes in 3D virtual worlds to help its cars' safety software expect the unexpected. It's called Gaussian splatting, and it can "create a vast amount of realistic, high fidelity 3D scenes and subjects from real world visuals." Volvo is tasking its spinoff software firm Zenseact to use AI to generate a range of on-road scenarios with this method. The company notes that objects in each of these environments can be manipulated and moved around to create rare "edge cases" of traffic situations and train driver assistance systems (ADAS). You can see an example scenario below. For example, you might have a lone pedestrian crossing the street in front of a car in one instance; in another, you could have that same person move quickly amidst a small group of slow-moving pedestrians. These can be used to test how the ADAS software will react in each situation. Alwin Bakkenes, who heads global software engineering at Volvo Cars, explained that this could give the automaker a way to prevent a far wider range of untoward incidents on the road. "We already have millions of data points of moments that never happened that we use to develop our software," he said. "Thanks to Gaussian splatting we can select one of the rare corner cases and explode it into thousands of new variations of the scenario to train and validate our models against." This new tech is powered by a supercomputing platform from Nvidia, which Volvo invested in recently. The Swedish marque has been in business with the chipmaker for several years now, integrating its compute solutions into its cars for advanced safety features. Current ADAS features typically rely on some combination of cameras, radar, LiDAR sensors, and ultrasonic sensors to detect obstacles and signals to navigate roads. The Gaussian splatting method could serve as another tool for Volvo to further enhance its cars' abilities to take over the wheel safely.

Volvo is Using Virtual Reality to Save Lives on the Road

Volvo is utilizing AI-generated, life-like virtual worlds to advance the development of its safety software, including driver assistance systems (ADAS), with the goal of making its cars even safer. The Swedish automaker can now synthesize incident data collected by the advanced sensors in its new cars, such as emergency braking, sharp steering, or manual intervention. This enables them to probe, reconstruct, and explore these incidents in new ways to understand better how they can be avoided. This is made possible by an advanced computational technique called Gaussian Splatting, which can generate many realistic, high-fidelity 3D scenes and subjects from real-world visuals. The virtual environment can be manipulated by adding or removing road users and altering the behavior of traffic or obstacles on the road to create different outcomes. Related How Nvidia's Cosmos Platform Could Change Transportation Nvidia's Cosmos platform could revolutionize transportation with AI-driven simulations, optimizing traffic, autonomous tech, and urban planning. Posts Such a technique allows Volvo to expose its safety software to all types of traffic situations at a speed and scale previously unattainable. The automaker can now develop software that performs effectively in complex, rare, yet potentially dangerous "edge cases," reducing the time it takes to expose their software to these scenarios from months to days. "We already have millions of data points of moments that never happened that we use to develop our software. Thanks to Gaussian Splatting we can select one of the rare corner cases and explode it into thousands of new variations of the scenario to train and validate our models against." - Alwin Bakkenes, Volvo's Head of Global Software Engineering One Part of the Puzzle Volvo utilizes virtual environments alongside real-world testing for software training, development, and validation due to their safety, scalability, and cost-efficiency. These virtual environments are developed in-house in collaboration with Zenseact, an AI and software company founded by Volvo. This project is part of a PhD program with leading Swedish universities to explore the potential integration of neural rendering techniques into future safety initiatives. The Wallenberg AI, Autonomous Systems, and Software Program (WASP) sponsored the study. A History of Using Data to Improve Safety Volvo has a long-standing history of using data and advanced technologies to enhance safety. Data collected by the Volvo Cars Safety Research team has been instrumental in developing and testing some of the world's most prominent safety features. In the 1970s, Volvo began leveraging data to improve safety, thanks to its Safety Research team. In the early days, the team would arrive at accident scenes with measuring tapes, evaluating skid marks and other crash indicators. The data and insights gathered from these accidents inspired numerous life-saving innovations, such as the Whiplash Injury Protection System and the Side Impact Protection System. Today, advanced technology allows them to use data even more effectively to prevent risky situations. Integration of NVIDIA Technology Thanks to its recently expanded partnership with NVIDIA, Volvo can explore technologies like Gaussian Splatting. The new generation of fully electric cars, built on NVIDIA-accelerated computing, collects data from various sensors to better understand what's happening in and around the car. An AI supercomputing platform powered by NVIDIA DGX systems contextualizes this data, unlocks new insights, and trains future safety models, enhancing and accelerating the development of artificial intelligence. This supercomputing platform is part of a recent investment by Volvo and Zenseact to establish one of the largest data centers in the Nordics. Source: Volvo Cars

Volvo is using AI-generated worlds to make its cars safer and it's all thanks to something called Gaussian splatting

The technique can recreate "edge cases" to train models faster Volvo is using a new AI technique called 'Gaussian splatting' to train its vehicles and accelerate its goal of zero collisions on the roads - and it's all thanks to its recently expanded partnership with Nvidia. Last month we reported that the upcoming Volvo ES90 will be the most powerful car it has ever created in terms of core computing capacity, due to it packing a dual Nvidia AGX Orin configuration. Now, the company has revealed how this sort of supercomputing is also helping it to more quickly train its Advanced Driver Assistance Systems. Volvo claims that it can now synthesize incident data collected by the advanced sensors in its latest vehicles, such as emergency braking, sharp steering or manual intervention. This then allows the company to reconstruct and explore them in new ways to better understand how incidents can be avoided. The novel method is dubbed Gaussian splatting and it allows the company's software to produce realistic, high-fidelity 3D scenes and subjects from real-world visuals. Once these scenes have been created, Volvo's engineers can manipulate them to generate a number outcomes. The video clip examples the Swedish marque provides are freakishly realistic. It's akin to a human learning how to skateboard by playing Tony Hawk's Pro Skater for hours and hours on end. "We can select one of the rare edge cases and explode it into thousands of new variations of the scenario to train and validate our models against," Alwin Bakkenes, Head of Global Software Engineering at Volvo Cars, explains. Bakkenes says this has the potential to unlock a scale that Volvo has never had before and even to catch edge cases before they happen in the real world. Gaussian splatting is a relatively new 3D rendering technique that doesn't rely on neural networks, unlike more complex methods such as Neural Radiance Field (NeRFS). This allows for incredibly complex 3D scenes to be created in real time. The technique is currently being explored in multiple industries, from gaming to interactive app development. Volvo's use of advanced Lidar, sensor and high-definition camera technology, as first showcased in the EX90, collects reams of data that can then be reproduced in a manipulatable 3D model, which allows its engineers to then train the vehicle's AI to perform better in the real world. There was some disappointment when the EX90 launched, seeing as its Lidar technology would remain offline for consumer use, effectively banished to merely collecting data until Volvo's compute power was at a level where the company was happy to introduce ADAS systems that rely on the sensor suite and software stack. Thankfully, its recently announced partnership with Nvidia will help the Swedish marque, which is synonymous with road safety, to realize its vision of zero collisions and driver assistance systems that actually help, rather than simply nag. What's more, the company has also stated that early EX90 models will be updated with the dual Nvidia AGX Orin System on a Chip set-up, so they too can make the most of the latest developments in autonomous driving and ADAS systems.

Volvo is using artificially generated worlds to improve car safety

It looks like Volvo has found a way to utilise artificial intelligence in a way we can all get behind. Using data gathered from its fleet of cars, the Swedish car maker is synthesising various situations and incidents to test and study its cars' safety features. This is all done through a new technique that is known as Gaussian splatting, and it uses 3D scenes and subjects from real world visuals to create a virtual world that can then be manipulated to Volvo's desire. From here, they can create a slate of road situations and behaviours to see how their cars would fare in different outcomes. The hope of this idea is that it will enable the company to test situations that would otherwise be too complex or rare to create physically, and thus allow them to begin creating counters and safety features that could further protect the driver and passenger from harm. Volvo's head of global software engineering, Alwin Bakkenes, stated: "We already have millions of data points of moments that never happened that we use to develop our software. Thanks to Gaussian splatting we can select one of the rare corner cases and explode it into thousands of new variations of the scenario to train and validate our models against. This has the potential to unlock a scale that we've never had before and even to catch edge cases before they happen in the real world." Gaussian splatting is powered by Nvidia technology, and even uses an AI supercomputing platform to study the data and provide insights so that Volvo can continue to lead the automotive sector as a safety pioneer.

Volvo is using AI to make its cars safer: How is it doing it? - Softonic

Volvo has long been synonymous with automotive safety, and now the company is leveraging artificial intelligence to take its crash prevention systems to the next level. Thanks to a groundbreaking partnership with Nvidia, Volvo is using Gaussian splatting, a cutting-edge 3D rendering technique, to train its vehicles to avoid accidents more effectively than ever before. Volvo is implementing Gaussian splatting to create ultra-realistic 3D scenes based on real-world driving data. By analyzing sensor data from its latest vehicles -- including emergency braking, sharp steering, and manual interventions -- the company can reconstruct and manipulate driving scenarios to improve its Advanced Driver Assistance Systems (ADAS). Unlike traditional AI training methods, this approach allows Volvo engineers to generate thousands of variations of rare "edge cases," ensuring the AI encounters and learns from high-risk situations before they happen on the road. This dramatically improves the system's ability to predict and prevent accidents. Volvo's collaboration with Nvidia is key to this advancement. The dual Nvidia AGX Orin system inside the Volvo EX90 and the upcoming ES90 provides supercomputing power capable of processing massive amounts of sensor data in real-time. This accelerates AI training, making autonomous and semi-autonomous driving more reliable and efficient. By harnessing this technology, Volvo is moving closer to its vision of zero collisions. While the EX90 initially launched without active Lidar functionality, future updates -- powered by Nvidia's advanced chipsets -- will unlock even more powerful safety features.