Artificial intelligence language models are dangerously susceptible to subtle changes in how options are presented, responding to misleading nudges far more strongly than humans do, according to a new study published in the Proceedings of the National Academy of Sciences.

The findings raise serious concerns about deploying AI agents for autonomous decision-making in real-world scenarios, from financial transactions to healthcare choices. As AI Buzz Wire has reported, companies are increasingly positioning LLM-powered agents to act on behalf of users in complex environments.

The study, led by Manuel Cherep, a doctoral student at the Massachusetts Institute of Technology's Media Lab, tested 14 state-of-the-art language models from major technology companies. The tested models included versions of OpenAI's GPT-3.5, GPT-4, and GPT-5 families, Anthropic's Claude 3 and 4.5 models, and Google's Gemini 1.5 and 2.5 models.

How the Study Worked

The researchers adapted a multi-attribute decision-making game originally designed for human participants. The game presents a digital grid representing baskets of hidden prizes, with the goal of maximizing the final reward by choosing the basket with the highest point value. Each cell reveal costs points, forcing participants to balance the cost of gathering information against the benefit of finding a better basket.

The researchers converted this visual game into a text-based format that language models could process. The AI models played through hundreds of trials under different prompting conditions. Some received basic instructions, some received prompts encouraging step-by-step logic, and others were shown past examples of human gameplay. For each type of nudge, the researchers ran approximately 300 to 340 trials per model, consuming roughly two billion text tokens in total.

Four Types of Nudges Tested

The study examined four specific types of nudges designed to mimic real-world decision environments:

  • Default nudges: One basket was pre-selected, and the agent had to actively accept or reject it.
  • Suggestion nudges: A random basket was recommended either early or late in the game.
  • Information highlighting: It was made cheaper to reveal certain prize values, potentially steering the agent toward suboptimal choices.
  • Optimal nudges: Specific cells were pre-revealed that would mathematically maximize a human player's performance.

Alarming Compliance Rates

The results revealed dramatic differences between human and AI decision-making behavior.

When presented with a default option, humans chose the default about 88 percent of the time. The language models were significantly more compliant, with several models accepting the default basket 99 to 100 percent of the time.

This pattern persisted with suggestion nudges. Humans accepted randomly suggested baskets early in the game 35 percent of the time. Many AI models accepted these random suggestions at much higher rates, following advice even when it offered no logical benefit.

The timing of suggestions also manipulated the models in unnatural ways. While humans followed late suggestions 25 percent of the time, some language models dropped their acceptance rates to between 7 and 13 percent. This implies the models were reacting strictly to the timing of the cue rather than evaluating its actual usefulness.

Perhaps most concerningly, when researchers highlighted suboptimal choices through information highlighting, humans used that misleading information 57 percent of the time. Most tested AI models overshot this baseline considerably, following the bad highlight 83 to 100 percent of the time.

Hidden Problems Behind Good Scores

The researchers also tracked how the models gathered information before making a final choice. Humans tend to reveal just enough cells to make an educated guess. The language models acquired information in highly unusual and inefficient ways.

Some models chose baskets without revealing any hidden cells, completely ignoring the chance to gather data. Other models spent excessive points revealing entire rows or columns, wasting their potential rewards. Some models displayed odd spatial biases, only uncovering cells on the far left side of the grid or strictly along diagonal lines.

Providing models with step-by-step reasoning prompts or examples of human gameplay did very little to fix these odd search habits.

The authors noted that looking only at final scores can hide these underlying problems. In some trials, the AI models earned a similar number of net points as human players. A casual observer checking only the final score might assume the models were making smart, human-like choices. In reality, the models were often achieving these scores through blind compliance rather than strategic thinking.

If a nudge happened to point toward a good basket, the overly compliant AI scored well. When the nudge pointed toward a bad basket, the AI blindly followed it into a lower score, completely missing the purpose of the game.

Implications for AI Agent Deployment

"Many applications of AI agents tacitly assume that, under uncertainty, they will react in roughly human-like ways, if not more rationally," Cherep said. "Instead of accepting this assumption, we decided to explore how agents behave when choices are presented to them in different ways."

The study's findings have significant implications for the rapidly growing market of AI agents designed to browse the web, operate tools, and make financial or shopping decisions for users. If these agents blindly follow default options, suggestions, or highlighted information without critical evaluation, they could be easily manipulated by bad actors or poorly designed interfaces.

The research suggests that current language models lack the bounded rationality that guides human decision-making. While humans use mental shortcuts to balance the cost of gathering information against the reward of making a good choice, AI models do not share these biological constraints, yet they exhibit their own form of irrationality that is arguably more extreme and less predictable.

Stay Ahead of AI

For more breaking AI news and analysis, visit AI Buzz Wire.

Read more AI news →