The Next Frontier of AI: From Language Intelligence to Social Intelligence

The recent history of AI can be told as a sequence of widening circles. First came systems that predict text. Then systems that follow instructions. Then systems that take actions, using tools in software environments. Each step changed what the technology could be used for, and each exposed a new kind of problem that the previous step had hidden. The next circle, we argue, is people: not as a source of text or instructions, but as other minds with goals, knowledge and boundaries of their own. This article sets out that argument and flags where it is interpretation and where it is evidence.

Step one to two: from text to intent

Making language models bigger did not automatically make them better at doing what users wanted. As the InstructGPT authors put it, large models can generate outputs that are untruthful, toxic or simply unhelpful, which means they are not aligned with their users. Their remedy was to fine-tune with human feedback: labeler demonstrations, then rankings of model outputs used for reinforcement learning. In human evaluations on the authors' prompt distribution, outputs from a 1.3-billion-parameter model trained this way were preferred to those from the 175-billion-parameter GPT-3 [1]. The lesson was that intent mattered more than scale for usefulness, and that people are needed to specify it.

Step two to three: from answers to actions

The second shift is from producing answers to completing tasks in environments. SWE-bench turned 2,294 real GitHub issues into tasks in which a model must edit a codebase to resolve the issue [2]. WebArena built functional websites for e-commerce, forums, software development and content management so that agents could be tested on realistic web tasks [3]. These evaluations differ from question answering: the model must plan, use tools and be judged on the resulting state of the world.

τ-bench added a human-like element, in which an agent holds a conversation with a simulated user while following domain rules and using tools, and is scored on whether the database ends up in the right state. Its authors reported that even strong agents succeeded on fewer than half the tasks and were inconsistent across repeated attempts [4]. This is already the border of the next step: once a user is in the loop, the environment includes a mind.

Step three to four: from environments to people

A person is a different kind of environment from a website. A person has beliefs that may be wrong, goals that may change, information the agent lacks, and preferences about privacy and tone. They are also not alone: many tasks involve several people with different information and interests.

Researchers have started to build tests for this. SOTOPIA places agents in role-played scenarios with social goals and evaluates them on a multidimensional framework; it found a hard subset of scenarios on which even the strongest model performed markedly worse [5]. Meta's CICERO showed what is possible when language is tightly coupled to explicit reasoning about others: by inferring players' beliefs and intentions from conversation, it achieved more than double the average score of human players in an online Diplomacy league [6]. And in cooperative text games, agents that were given explicit representations of what their partners believed performed better and made more accurate inferences [7].

Generative agents showed another direction: memory, reflection and planning allowed software agents to display believable social behavior in a simulated town [8]. Believability is a different property from accuracy about real people, but the architecture, with a persistent memory that the agent reflects on, points at something that real social assistants will need.

What changes in the move to social intelligence

Four things are different.

1. Other minds are partly hidden. The agent has to infer beliefs and goals, and be honest about uncertainty. Theory of mind is the name for part of this.
2. Time matters. Relationships unfold over months. Most evaluations use a single conversation, which means long-term memory, promises and changing context are largely unmeasured. This is a gap we flag in Research Report 001.
3. Success is partly subjective. For software tasks, tests can verify results. For tact, consent or fairness, qualified people can disagree, which pushes evaluation toward rubrics, multiple judges and recorded disagreement.
4. Cooperation becomes the goal. Dafoe and colleagues argued for a research agenda in cooperative AI: equipping machines with capabilities to cooperate and to foster cooperation [9]. Many of the most valuable social tasks, such as scheduling among several people, negotiating, mediating and introducing, are cooperative problems.

Why this matters for products

A product that helps people connect, whether by scheduling, introducing, drafting or reminding, is only as good as its model of the people involved. It needs to know who may be told what, what a person would be uncomfortable with, and when to ask instead of act. These are the capabilities that our consumer product HUMAN depends on, and why HUTE does research and builds infrastructure alongside the product. See HUTE and HUMAN.

Where we may be wrong

The "widening circles" story is a simplification. The categories overlap, and progress in one is not strictly required before another. It is also an interpretation of a field that is moving fast: models evaluated in the cited papers have already been superseded. What we are confident of is the narrower point that the open problems have shifted, from "can the model produce the right text?" to "does it represent the people it is dealing with, and act appropriately in light of that?" For how we think about measuring that, see reinforcement learning for socially intelligent agents and Research Report 002.