NEC Laboratories America
INTEGRATED SYSTEMS
PROJECTS
PEOPLE
PUBLICATIONS
PATENTS
A great deal of essential work is still done manually, by people working directly in the physical world. Much of it is repetitive, some of it is hazardous, and it increasingly changes from one job to the next over short periods of time.
Traditional robots have struggled with this kind of variable work.
They excel at fixed, repetitive tasks in carefully engineered environments, but adapting them to something new often requires reprogramming the robot, redesigning fixtures, and reconfiguring the surrounding workspace. The cost and time of that integration can outweigh the value of automating the task in the first place.
Keyword Tag: physical ai autonomy
Physical AI creates the possibility of changing this equation. Advances in vision-language-action (VLA) models, world models, and multimodal reasoning are enabling robots to understand their surroundings, follow more general instructions, plan over longer horizons, and perform increasingly complex tasks. Instead of engineering the environment around a fixed robot program, the robot can begin to adapt its behavior to the environment and task. This opens the door to automation in far more flexible settings: factories that change what they produce, warehouses with constantly evolving workflows, and other industrial environments where robots must perform changing tasks alongside people.
But greater autonomy creates a new challenge. In the physical world, mistakes have consequences. An incorrect answer from an information system can often be regenerated. An incorrect action by a robot can damage equipment, interrupt operations, ruin a product, or create a safety risk. The bottleneck is therefore shifting from making Physical AI more capable to making that capability dependable. Flexible robots must know how to act when conditions differ from training, how to detect when something has gone wrong, how to recover safely, and how to operate predictably around people.
We are building toward dependable autonomy for Physical AI: systems that combine the flexibility of modern AI with the reliability required for real-world deployment, allowing robots to operate safely and effectively in environments that are uncertain, changing, and shared with people. Three research challenges stand between increasingly capable Physical AI and dependable real-world autonomy: runtime assurance, continual adaptation, and human-robot collaboration.
First, runtime assurance asks how an autonomous system can recognize when its assumptions, plans, or actions are becoming unreliable. This layer also intervenes before a small error becomes a consequential failure. In an uncertain physical environment, a robot cannot assume that the world will behave exactly as its model predicts. It must continuously evaluate whether execution is proceeding as expected and know when to adjust, stop, or seek help. Second, continual adaptation asks how a system can respond to unfamiliar objects, tasks, environments, and execution outcomes without being retrained every time something changes. Physical environments are inherently variable, so useful autonomy must adapt during operation rather than depend on every possible condition being anticipated in advance. Third, human-robot collaboration asks how people can naturally guide, constrain, correct, and work alongside increasingly autonomous machines. As robots take on more complex tasks, people need intuitive ways to communicate intent, establish boundaries, provide corrections, and step in when autonomous execution is not enough.
These challenges define the gap between capability and dependability. Foundation models, vision-language-action models, and world models are rapidly expanding what autonomous systems can understand and do. But capability alone does not provide the mechanisms needed to operate reliably when the physical world departs from what those models expect.
Our AI systems have a layer for dependable autonomy around frontier models, rather than requiring the models themselves to solve every reliability problem. This layer maintains and updates knowledge about an uncertain and changing world, monitors plans and actions as they execute, detects when assumptions or behavior diverge from expectations, and adapts execution when conditions change. It also provides mechanisms for enforcing operational constraints and incorporating human guidance when autonomous execution alone is insufficient. The result is an architecture in which increasingly powerful AI models provide intelligence and flexibility, while a surrounding systems layer provides the monitoring, adaptation, constraints, and human interaction needed for dependable operation. Rather than competing with the effort to build ever more capable VLA and world models, we address the complementary systems question: how do we turn increasingly capable Physical AI into autonomy that can be trusted to operate in the real world?

