Mercedes Unleashes AI-Enhanced Conversational Navigation Powered by Google Gemini

Engaging in conversation with your vehicle is becoming standard.

On Monday, Mercedes-Benz declared a collaboration with Google to incorporate AI-driven conversations into the integrated navigation system. The Automotive AI Agent (part of Google Cloud) will have the ability to identify destinations based on precise inquiries alongside charging stations and other navigation necessities.

The forthcoming CLA series will be the inaugural model equipped with Mercedes’ innovative MB.OS operating system, which includes the latest Google-supported AI features. Engaging in natural dialogue with the MBUX Virtual Assistant will now source information from Google’s AI service, Gemini, enabling users to pose more conversational inquiries to Google Maps, such as, ‘Where is the nearest restaurant with vegan choices?’ You can then ask follow-up questions like, ‘What’s the highest-rated dish there?’ or ‘What time does the restaurant close?’ after a suggestion appears. Navigation directions will display on the front screen.

This CLA series is set to be the first to provide this AI-driven navigation support later this year.

At Mercedes’ Research & Development center in Sunnyvale, California last week, CEO Ola Källenius elucidated how the partnership functions. “We architect our software stack,” he stated. “Google provides the cloud services.” Källenius assured that the Gemini functionality would remain complimentary for Mercedes owners, although potential subscription models may be considered for the full suite of AI and additional technological offerings.

Mercedes customers will remain within the Mercedes digital ecosystem mainly through the Mercedes me app, eliminating the need to log into Google to access the generative AI capabilities in Google Maps, which is already utilized for navigation.

While ChatGPT via Microsoft Azure OpenAI Service was recently integrated into the MBUX Voice Assistant system last month for more general knowledge inquiries following a 2023 beta with the AI chatbot, this new incorporation aims to enhance the “info” in “infotainment.”

Källenius emphasized that the goal is seamless integration. “It needs to be incredibly intuitive to use,” he remarked. “You should be able to simply converse with the [gen AI].” The traditional “wake word” of “Hey, Mercedes” will serve as the gateway to the service. Ultimately, you’ll be able to start speaking to the car, and it’ll recognize that you seek information, but for now, you need to alert the vehicle. Additionally, users can press a button on the steering wheel or utilize the touchscreen to engage the AI assistant.

As AI assistance becomes increasingly proficient in the vehicle, it has developed from merely processing basic, solitary inputs adhering to structured inquiries to understanding multiple requests within a single question while employing natural, everyday language (“Hey, Mercedes, where can I celebrate my mom’s 80th birthday that isn’t overly noisy but also provides a children’s menu for the grandkids?”).

Conversing with the vehicle through Google is initially debuting with Mercedes, but the German manufacturer acknowledges that other brands will soon follow suit and potentially learn from the Mercedes-Google partnership.