A self-driving car must continually decide when to accelerate, slow down, pass or wait. Wayve’s Lingo-1 is designed to make part of that process visible: the model combines visual information with text-based logic, producing descriptions of the road situation and explanations for its choices.
Giving driving decisions a spoken rationale
Many autonomous driving systems use visual perception to guide their decisions. Lingo-1 adds a language layer between perception and action. As the car encounters traffic, it can generate statements about what is happening and why it is responding in a particular way.
Wayve compares this to a driver thinking aloud or an instructor helping a learner pay attention to the road. The idea is to give people a clearer account of the system’s interpretation, so its decisions may feel less like a black box. An explanation can make a choice easier to follow, though it does not by itself establish that the choice is correct.
The language layer is also intended to help the system reason through situations that were not part of its training data. Wayve says causal reasoning matters because autonomous vehicles need to understand relationships between objects and actions in a scene, rather than simply recognize what is visible.
Training with descriptions as well as images
Lingo-1 was trained on image, voice and action data gathered by Wayve drivers as they drove around London. The company says its approach can also use examples written by people to adjust how the system should behave. That could reduce the need to collect extensive visual data for every situation.
For instance, Wayve says a small set of examples describing how a car should respond to a pedestrian, and which factors to consider, could take the place of thousands of visual examples of braking for a pedestrian. In this approach, text descriptions provide guidance about the scene and the desired behavior.
Wayve also points to the general knowledge held by large language models. It says these models have learned about human behavior from internet-scale datasets and can distinguish concepts such as a tree, a shop, a house, a dog chasing a ball and a bus stopped in front of a school. That knowledge could help a driving model interpret unfamiliar scenes and common driving concepts.
Progress and limits
According to Wayve, Lingo-1 currently achieves 60 percent of the accuracy of human drivers. The company says its performance has more than doubled since initial testing in August and September, following changes to its architecture and training dataset.
The result comes with important limits. Lingo-1 has been trained only on data from London and the UK, so the source does not establish how it would perform in other places. Like other language models, it can generate incorrect answers. Wayve says its grounding in real-world visual data is an advantage, but that does not remove the possibility of errors.
There are also technical challenges to address. The model needs long context lengths to describe video, and Wayve identifies integrating Lingo-1 directly into an autonomous vehicle’s closed-loop architecture as another challenge. That means the approach still has practical hurdles between producing a textual rationale and operating as part of a vehicle’s driving system.
How Lingo-1 fits Wayve’s other work
In June, Wayve introduced GAIA-1, a generative AI model intended to help with the limited supply of video data for training systems across different traffic situations. GAIA-1 learns driving concepts by predicting the next frames in a video sequence.
The two models address different parts of the training challenge described by Wayve. GAIA-1 can help generate video sequences for learning about road scenarios, while Lingo-1 adds language-based descriptions and reasoning to driving decisions. Together, they reflect the company’s effort to help autonomous systems learn from both visual material and explanations of what is happening on the road.