SRI is discovering how AI can organize humans and machines into complex, collaborative, high-functioning units.
From factory floors to roadways to software engineering, humans are learning to work alongside highly capable automated systems. But where others see unmitigated success, SRI senior computer scientist Aswin Raghavan sees an emerging problem.
“When it comes to task automation, a lot of what we see now is someone saying: Okay, here is a task that I believe is within the competency of this robot or this AI model,” observes Raghavan. “If it fails, all I want to do is detect that it has failed and assume that there is a human on hand to help.”
For high-stakes scenarios like troop movements, air traffic control, and expensive manufacturing processes, that cavalier attitude toward failure doesn’t work. Rather, humans and their machine collaborators need to communicate with and adjust to one another in real time. If they don’t, a single error can cascade through the entire team and undermine the mission at hand.
As the principal investigators for an internally funded SRI project called Machine-Assisted Coactive Design, Raghavan and his colleague Saurabh Farkya are seeking to design better human-machine teams that can handle complex, real-time teaming situations, whether in a smart factory or on a battlefield.
Why automating human-machine team design matters
Today, Raghavan observes, even the most state-of-the-art work on human-machine teaming largely assigns tasks by hand. This stream of research — called “coactive design” — struggles to scale to complex human-machine teams. Human designers simply can’t process all of the variables that impact the performance of large human-machine teams.
“What if you leveraged AI to organize the team and assign the tasks starting from first principles, rather than basing those decisions on human assumptions about which tasks make the most sense to hand over to a computer?” — Aswin Raghavan
SRI’s new work proposes a fundamental paradigm shift: What if a carefully architected and trained AI system could design human-machine team structures that outperform anything we could sketch out on our own?
As troops operate alongside high-tech drones and factories become a mesh of real-time human decisions and lightning-speed automated activity, starting with the right team structure is paramount. An AI-driven system could take into account both the larger mission and the strengths and weaknesses of each team member (whether they are a human, a robot, or an AI agent) and design a team structure that fully anticipates how all of the entities will interact in real time.
Solving the team design problem
Any viable human-machine team, Raghavan explains, will be defined by three characteristics: observability, directability, and predictability. All participants (human and otherwise) need to be able to observe what others are doing, direct each other as necessary, and predict the behavior of their collaborators.
But creating those conditions is about much more than applying some educated guesses about the tasks that make sense for a machine.
“What if you leveraged AI to organize the team and assign the tasks starting from first principles, rather than basing those decisions on human assumptions about which tasks make the most sense to hand over to a computer?” Raghavan asks.
“Our approach is capable of optimizing over a large number of possible teams. For multiple reasons, it seems to be a better way to design complex human-machine teams.” — Aswin Raghavan
Large language models (LLMs) are critical — they allow an automated team-design system to reason about how teams will communicate and act in concert. But, Raghavan cautions, approaches based exclusively on LLMs tend to perform poorly in terms of observability and directability. Because of this, SRI’s approach —leveraging a shared “doctrine” that constrains the LLMs — is much more likely to yield automated coactive design that functions in the real world, delivering team structures characterized by factors like robustness to failure and built-in redundancy.
“Doctrine is something that all of the agents and humans in the team are aware of. It creates more predictability than in a standard coactive design approach,” says Raghavan. “Furthermore, machine-assisted coactive design uses simulation to optimize for task success in an automated manner. Our approach is capable of optimizing over a large number of possible teams. For multiple reasons, it seems to be a better way to design complex human-machine teams.”
The first year of the project, explains Raghavan, largely focused on collecting and honing the datasets used for model training. In year two, the team successfully demonstrated coactive design using an LLM (rather than a manual process) to design teams and assign tasks. Last year, the team focused on using simulations to further optimize the system’s decision-making, conducted a user study with internal SRI participants, and continued to publish and present new research about this unique human-machine teaming paradigm.
Moving from small teams to big teams
“Most of the research right now is focused on small teams,” says Raghavan. “There’s very little work on multiple humans controlling multiple agents.”
But it’s clear that the future of human-machine teaming will depend on big teams. In a military context, modern command and control demands a course of action (COA) that allows both troops and intelligent, AI-equipped hardware platforms to survive friction, disrupted communications, and enemy pressure. SRI’s work on machine-assisted coactive design demonstrates how AI can help military planners design COAs that are structured for real operations: decomposing missions into interdependent tasks, aligning the right human and machine teams to each task, and enabling units to act coherently even when they are forced to operate with less centralized control.
Raghavan and Farkya are also examining air traffic control as a prominent future use case for automated coactive design. Over the years, painstaking design work has created an air traffic control system that’s highly capable but still not perfect. As our airspace becomes crowded with delivery drones and air taxis, managing airspace will keep getting harder.
“That’s exactly the kind of situation where automated coactive design could help,” Raghavan observes. “Even if some decisions remain manual, automated coactive design could provide human supervisors with numerous new and counterintuitive ideas about how to optimize their human-machine teams.”
Explore how SRI is innovating in AI and security.
Elements of this project were presented at the Department of Defense Basic Research Conference under the title “Mitigating Hallucination in LLM-Based Task-Team Design with Submodular Optimization: Application to Command and Control at Higher Echelons” and at the International Command and Control Research Technology Symposium (ICCRTS 2025) under the title “COA Generation in the Age of LLMs.” The inventions herein are associated with U.S. Provisional Patent Application No. 63/674,603.
