Designing LLMs as collaborators, not standalone controllers

What they studied

The authors examined how large language models could support collaboration between people and automated vehicles. Their goal was to develop a human-centered framework for understanding the roles these models could play, rather than treating an LLM as a complete driving system.

Following PRISMA guidelines for systematic reviews, they reviewed 1,126 records published between 2021 and 2025. After screening and full-text assessment, they included 52 studies. This paper presents a synthesis and design framework, not a reported test of a new autonomous-driving controller.

What they found

The review identified four recurring groups of capabilities: perception and awareness, reasoning and decision-making, action and control, and interaction and communication.

Across these functions, the authors found promise in using LLMs to interpret the meaning of driving scenes, explain decisions, support high-level planning, and communicate with drivers in both directions. These roles emphasize understanding and collaboration, rather than simply issuing control commands.

The abstract also identifies substantial limitations: hallucinations, incomplete physical grounding, non-deterministic reasoning, and latency. In plain terms, concerns include producing incorrect information, lacking a complete connection to the physical situation, reasoning inconsistently, and responding too slowly. These limitations matter particularly in safety-critical settings.

The synthesis suggests that LLMs are most effective as high-level cognitive partners integrated with verified, task-specific modules, rather than as standalone controllers.

Why it matters for builders

For someone building an LLM-powered application, the useful architectural question is where the model’s responsibility should end. This framework offers a way to separate interpretation, planning, and communication from tasks that require verified specialist components.

That is a design lesson to consider, not proof that the same architecture will improve every app. The review concerns human-vehicle systems, not AI coding assistants or general software development. Its practical value is a clearer way to think about model responsibilities and limitations when designing collaborative AI features.

Source: "From Code to Collaboration: A Cognitive Agent Framework for Large Language Model (LLM)-Based Human-Vehicle Teaming" (2026-09-03, Proceedings of the Human Factors and Ergonomics Society Annual Meeting). This is our summary of the paper's abstract — read the original for full details.