Google Gemini taps Volvo EX60 cameras to interpret surroundings and control car features

Google Gemini Brings Visual Intelligence to Volvo EX60

At Google I/O 2024, the tech giant unveiled a significant leap forward for AI in cars by demonstrating how Google Gemini can access the Volvo EX60's external cameras to process real-time visual information and interpret its surroundings1

1

2

2

. During live demonstrations, the system identified San Francisco's Transamerica Pyramid using the car's front-facing camera, explaining that it "stood as the tallest building in San Francisco for 45 years" while also noting nearby landmarks like the historic Sentinel Building1

1

. The feature effectively transforms the vehicle into a rolling version of Google Lens, deeply integrated through Android Automotive and Volvo's new HuginCore software architecture3

3

.

The Gemini AI features in cars extend beyond visual recognition. During demonstrations, voice commands allowed users to translate road signs written in foreign languages and even control vehicle functions like making the electrochromic sunroof opaque to block harsh sunlight1

1

. This multi-modal understanding relies on computing hardware from Qualcomm and Nvidia, combined with the EX60's neural processing engine2

2

.

Practical Applications Through Google Built-in and Android Auto

Google Built-in, the native operating system for car infotainment systems, enables direct access to Google's services without requiring a phone connection. Demonstrations showed Gemini identifying dashboard warning lights and calculating whether a 65-inch TV would fit in the vehicle's cargo area1

1

. Meanwhile, Android Auto features showcased on the Kia EV9 included personalized dashboard designs built on Google's Material 3 Expressive, allowing drivers to display custom widgets alongside photos1

1

.

The context-aware AI also simplifies daily tasks to maintain driver focus. Magic Cue pulls information from email and calendar apps to respond to texts automatically—during the demo, it answered "What time is the pottery class?" by finding the details: June 1 at 2 p.m.1

1

. For meal planning, voice commands enabled ordering two fish tacos from Pacific Catch through DoorDash, with Gemini navigating the app and adding items to the cart like a human would1

1

.

Enhanced Navigation and Entertainment Features

Vehicles supporting both Google Built-in and Android Auto compatibility will gain access to 3D Immersive Navigation for Google Maps, displaying buildings, overpasses, terrain, lanes, traffic lights, and crosswalks in three dimensions1

1

. The Volvo EX90 and ES90 will join the EX60 as the first models receiving this feature2

2

. Additionally, YouTube Music integration and the ability to watch YouTube videos in 4K resolution at 60 frames per second while parked—automatically switching to audio-only when driving—will enhance the charging experience1

1

.

Privacy Concerns and Future Rollout

While Google emphasizes that the camera feature activates only with driver permission and blurs faces during image analysis2

2

, privacy concerns have already emerged online. Some users questioned data collection practices, with one Reddit commenter asking "How do I disable that?" while another observed that modern cars are becoming "phones on wheels"3

3

. The project remains unreleased publicly, with both companies tight-lipped about the exact rollout timeline, though Google confirmed the features will arrive later this year1

1

2

2

. This expansion represents Google's broader push to integrate Gemini across cars, phones, wearables, and smart glasses, fundamentally changing how drivers interact with their vehicles through natural voice commands.