Tell a robot "don't spill coffee on my laptop" and today's machines will do exactly that — and nothing more. The phone sitting next to the laptop? The tablet charging on the other side of the table? Off the robot's radar until you spell out each one, correction by tedious correction.
A new platform from MIT and New York University researchers aims to close that gap. The system, called FLEX, enables motion-planning robots to follow a human's explicit instructions while also drawing broader generalizations from them that improve performance, according to MIT News. The team will present the work at the annual Conference on Robot Learning (CoRL) this month. For more context on this story, see our ongoing AI news.
"LLMs have seen so many texts from the internet," said Nathan Dennler, a postdoc in MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) and a study author. "People might have written about, 'Oh, my phone dropped in the water and now it doesn't work,' or 'I spilled coffee on my computer and now it doesn't work.' So LLMs pick up on the idea that liquids plus devices leads to things not working."
From One Warning to a Whole Rulebook
In one demonstration scenario described by MIT News, a person watches a robotic arm carry a coffee cup across a cluttered table and types a single command: "Don't spill on the laptop!" A conventional robot would add the laptop to its avoidance list and replan its path. FLEX goes further — the robot reasons about why laptops matter and adjusts its trajectory to avoid nearby electronics, including a phone the user never mentioned.
"The idea is that there is some higher-level intention behind the words that you say," Dennler said. "People naturally get this. But you have to carefully explain that to a robot, which takes time and effort. FLEX gives you an interface of communication with a robot to say, 'When people say this, they mean that.'"
The study was led by Andreea Bobu, an assistant professor of aeronautics and astronautics at MIT who heads the Collaborative Learning and Autonomy Research (CLEAR) Lab. Co-authors include Audrey Lee, who completed her SM at MIT in 2026, and Ilia Sucholutsky of New York University.
How FLEX Works: Add, Generalize, Mask
FLEX operates as a three-step pipeline that runs in roughly five seconds.
Step 1: Add
When a user issues a natural-language correction, the robot adds the relevant feature to its running list of things that matter for the task. If a person says "Don't drop it on my mug!" while the robot carries a bowling ball, the mug joins the avoidance list and the robot updates its trajectory. This step reflects how previous methods have always worked — and it is where their trouble begins.
"The problem happens when your feature list becomes super long, and some features are no longer relevant because your environment may have changed," Dennler told MIT News. A robot told to avoid every object individually becomes slower to correct and slower to plan, and the burden of enumeration falls on the human.
Step 2: Generalize
This is FLEX's core contribution. A large language model — GPT-4o, in the team's implementation — examines the accumulated feature list and searches for patterns among the items. Faced with a mug plus other fragile objects the person has flagged, the LLM infers a general class: avoid fragile things, even ones never mentioned. Because the model trained on vast libraries of human text, much of everyday common sense arrives pre-installed.
"LLMs have seen so many texts from the internet, from all sorts of people and places," Dennler said. "Because they've trained so much on those things, some of that common sense gets absorbed into the LLM."
Step 3: Mask
Real environments change, and stale rules cause erratic behavior. FLEX's third step uses the robot's standard computer vision system to determine which flagged objects are no longer present, then deletes those entries from the features list — keeping the robot's constraints current without human intervention.
Tested in Coffee, Candle and Bowling Ball Simulations
The team evaluated FLEX in simulations of a robot arm performing three tasks: serving a cup of coffee, moving a candle, and carrying a bowling ball. Volunteers interacted with the virtual robot by typing natural-language commands such as "Don't spill on my laptop," "Stay away from the stack of paper," and "Don't drop on the mug."
In all three scenarios, the robot followed each command literally while also reasoning at a higher level — steering clear of electronics when warned about the laptop, avoiding flammable objects when told to watch the candle, and treating all fragile items as no-drop zones after a single mug warning.
Why Common Sense Is the Bottleneck for Home Robots
The research targets a persistent obstacle in consumer robotics: nobody ships a robot that already knows what matters in your home. Physics-based trajectory planning is precise but opaque, and expecting users to manipulate mathematical cost functions is a non-starter.
"Robots use math to do their trajectory planning. But we can't expect a consumer to learn to do this math in order to tell a robot what to do," Dennler said. "Instead of having to get people to learn how to use a robot in a specific way, the goal is to interact with the robot through natural language."
A large strand of robotics research already weaves language into robot learning — clarifying task goals, redirecting trajectories, defining safe paths. But MIT News notes those existing methods use language mainly to tell the robot how to physically perform a task. Bobu's group is pushing on a different frontier: extracting a person's underlying, unspecified preferences from a handful of casual commands.
"Everyone's homes are different, and everyone cares about different things, so no robot comes out of the box knowing what matters to you," Bobu told MIT News. "What excites me about FLEX is that it can pick that up from a sentence or two instead of asking you to spell out every rule — and that's the difference between a robot people put up with and one they actually want in their home."
What Comes Next
The work, detailed in the paper "Flexible Language-Guided Representation Editing for Adaptive Robot Behavior," was supported in part by the MIT Generative AI Impact Consortium and the MIT Schwarzman College of Computing's Social and Ethical Responsibilities of Computing program.
The results shown so far are simulations, and translating that performance to physical hardware — with messier perception, noisier language and real consequences — remains the field's next hurdle. But the direction is clear: as robots move from factory cages into kitchens and offices, the machines that succeed may be the ones that understand not just what we say, but what we mean.
This article is based on reporting by Jennifer Chu, published by MIT News on November 9, 2026.---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
