Developing a Personal Assistant for Autonomous Driving (PASAD) Prototype

July 27, 2026 – Reading time: 5 minutes

What if your car had the ability to understand and accurately respond to your questions and requests? This was answered by our team’s contribution to the SALSA Project, a project aimed at increasing the safety and acceptance of automated vehicles in mixed traffic. Through the development of a Personal Assistant for Autonomous Driving (PASAD) prototype, they were able to learn what it takes to build the technology and its impact in the larger research landscape of AI and autonomous driving.

In this article, we give insights from the team that worked on INVENSITY’s contribution to the project.

Autonomous vehicles are gaining more ground, yet one challenge remains at the center of the discourse: how passengers can understand and trust the decisions made by the vehicle. The car’s sensors, control systems, and the facial and voice recognition software can get better over time, but the passenger’s experience will ultimately depend on the car’s ability to communicate in a way that understands emotions, worries, and interests like it was a good conversational partner.

As a research partner in the SALSA Project, this challenge was taken on by our research team through the development of a Personal Assistant for Autonomous Driving prototype.

The PASAD prototype is an AI assistant that can act and communicate with people based on their background, technical knowledge, communication preferences and actual emotional state. It is able to appropriately respond to people who have deep or surface-level knowledge of cars, as well as those who are calm or stressed when placed in unexpected situations. Regardless of who the assistant is addressing, the passenger should be communicating with an AI that knows and understands them.

Fabian Ziegler, Project Head of the PASAD prototype team, provided a hypothetical but realistic scenario of how the AI assistant should behave: «There’s a driver, let’s call her Alice, and the PASAD knows she is familiar with how autonomous driving systems work. While on the road, Alice feels upset and the system takes note of this based on her word choices, tone of voice, and other factors. When the car needed to brake because there was an animal in front of the vehicle, the PASAD does not make a long explanation and instead provides a brief apology and reason. Something along the lines of ‘Sorry for the braking, a Minimal Risk Maneuver was necessary,’ in an unprovocative way to not further upset Alice.»

When asked why the INVENSITY team worked on creating the PASAD prototype, Fabian explains that personalized communication is a crucial part of normalizing the technology in society. Because people can have different emotional states and communication styles, it is important to have a vehicle assistant capable of detecting these and adapt appropriately.

Another reason is that, in Europe, the restrictions for technologies involving AI or automation are very comprehensive, as exemplified by the General Data Protection Regulation (GDPR) and the EU AI Act. A key feature of the prototype was having the data stored locally or on premises. This means that third parties would have no access to the passenger’s data, protecting their privacy.

Lastly, by developing the AI assistant, it can serve as a foundation for more advanced AI assistants outside of automotive like in the medical industry where it can support the elderly or mentally challenged persons.

Developing an AI assistant that operates independently of cloud infrastructure or proprietary technologies is a worthwhile goal, but it comes with practical and technical challenges.

According to Adji Arioputro, a key researcher in the project, one key hurdle that the team faced was developing a working prototype with on-prem hardware to simulate air-gapped situation. Local software development was also a challenge since the team could not reuse the proof-of-concept application which was fully developed in cloud platforms. However, they were successful in implementing the RAG system with open-weighted models such as Ministral-3, as well as access to OpenAI’s Whisper voice model.

The contributions of the INVENSITY team, as well as other research partners, show that innovations in AI and automated driving can start from small but meaningful efforts. At the Half-Time event, one researcher developed a camera and their own algorithm to check a passenger’s emotions or expressions. Another generated the different postures or physiological situations a person can have.

Both Fabian and Adji expressed their amazement of their fellow researchers and sense of responsibility for developing human-centered technology.

As Adji says, «If we want to build a world where it’s safe to drive on the road, you have to work on the small things. This is important for building transparency and the public’s knowledge of AI – something we need as AI becomes more ubiquitous.»

Adji Arioputro is an AI Solutions Architect whose expertise lies in application development, software engineering and its intersections with other fields. He has worked for the company for almost 10 years, taking on different roles such as software developer, test architect, DevOps consultant and process engineer.

Dr. Fabian Ziegler is a Business Manager responsible for more than 15 active projects and the current Project Head for INVENSITY’s contribution to the SALSA research project. His previous work includes working as a consultant and system architect for several sensor systems, later becoming an MBSE expert leading several high-level consulting projects.

Author

Contributors

Resources