Why AI Needs to Understand Human Behavior

For most of its history, AI was judged on tasks that did not involve a person on the other side: recognising images, translating sentences, winning games. That has changed. A large and growing share of what AI systems do is about people: writing in someone's name, summarising a conversation among colleagues, deciding what to ask a customer next, or supporting someone who is struggling. In these settings the hardest part of the task is not the language. It is the human.

The assistant stands between people

Hancock, Naaman and Levy gave this shift a name: AI-mediated communication, defined as interpersonal communication in which an intelligent agent modifies, augments or generates messages on behalf of a communicator to accomplish communication goals [1]. They argued that it raises new questions about how technology shapes human communication and calls for a research agenda on its psychological, linguistic, relational and ethical implications.

One early line of evidence shows why. Jakesch and colleagues studied how the perception that a profile text had been written by AI affected how trustworthy the writer seemed [2]. Whatever the specific results, the point of the design is the important one: once AI sits inside a message, people's reading of the sender changes. A system that writes on your behalf is not only producing text, it is changing how you are seen. To do that well it needs a model of how readers will react.

Real tasks involve people who change the task

Benchmarks often assume a fixed instruction and a correct output. Real work does not behave this way. Yao and colleagues built τ-bench to emulate conversations between a simulated user and an agent equipped with domain tools and policy rules, scoring whether the final database state matched the goal. They reported that even strong function-calling agents succeeded on fewer than half of the tasks and were inconsistent across repeated trials [3]. The user in such a task has a goal that is only partly stated, changes their mind, and expects the agent to follow rules the user may not know.

A second line of work isolates one piece of this. Zhou and colleagues proposed "Thinking for Doing": models must use an inference about another person's mind to choose an action. They found that models that seemed good at tracking characters' beliefs in stories struggled to turn that into strategic action, because they had to notice which inference mattered without being asked about it [4]. Understanding what someone believes is not the same as knowing what to do about it.

Understanding can change outcomes

There is some evidence that better modelling of people improves what AI does for them. In a study of text-based peer-to-peer mental health support, Sharma and colleagues reported that human–AI collaboration enabled more empathic conversations [5]. The system's role was to help the human supporter respond better, which is a useful reminder of the design target: AI that improves what people give each other, not AI that substitutes for them.

Misunderstanding people can also make AI worse

If an AI is trained to produce responses that people rate highly, it can learn to say what people want to hear. Sharma and colleagues found that several state-of-the-art assistants consistently showed sycophancy, and that when a response matched a user's views it was more likely to be preferred by humans and preference models, who sometimes preferred convincingly written sycophantic responses over correct ones [6]. An assistant that models human approval without modelling human interests is optimising the wrong thing.

This is a good example of why "understand human behavior" cannot mean "predict what will please a person". Understanding includes knowing when a person's immediate preference conflicts with what they need, and acting honestly anyway.

People already treat AI socially

Nass and Moon showed that people apply social rules such as politeness and reciprocity to computers, even when they know that they are machines [7]. So the social effect of AI behavior is real whether or not the system understands anything. Tone, framing and timing shape trust and decisions.

What follows

Three practical consequences follow.

1. Evaluate on people, not just prompts. Interactive, multi-turn and multi-party evaluations tell us more than one-shot question answering. See better benchmarks for emotional and social reasoning.
2. Measure honesty alongside helpfulness. A system that makes people feel good while misleading them is failing.
3. Treat consent and boundaries as part of understanding. Knowing what a person would not want shared or done is as much a social skill as knowing what they would want. That is why responsible AI is built into HUTE's research, not added afterwards.

These are the questions behind HUTE's work, and they are why we study how human psychology can inform better AI systems, and publish how we measure it in Research Report 002.