The State of Social Intelligence in Artificial Intelligence: Capabilities, Limitations, and Open Problems

Abstract

Social intelligence, the ability to understand other people and act sensibly among them, is central to the situations in which AI systems are now deployed: drafting messages, mediating communication, negotiating, coordinating and offering support. This report reviews the published evidence on how well current AI systems do these things. We organise the literature around six areas: theory of mind and mental-state reasoning, emotion and empathy, pragmatics, interpersonal and multi-agent behavior, evaluation, and cultural generalisation. We then review safety risks that arise specifically because a system is socially capable.

Our reading of the literature, which we mark as interpretation, is that three things can be said with reasonable confidence. First, large language models now solve many classic social-cognition tasks that earlier models failed. Second, that success is fragile: small changes to a task can reverse it, and the field disagrees about what it shows. Third, measurement is the binding constraint. Benchmarks for social reasoning have uncertain construct validity, are vulnerable to contamination, are rarely validated across cultures, and often rely on a single "correct" answer where reasonable people disagree. We close with a list of open problems.

1. Scope and research questions

We ask four questions.

1. Definition. What is meant by social intelligence, and how does it relate to theory of mind and emotional intelligence?
2. Capability. What do published evaluations show about the ability of AI systems, mainly large language models (LLMs), to reason about mental states, emotions, social norms and other people's goals?
3. Limits. Why do reported capabilities conflict, and how reliable are the measurements?
4. Risk. What new risks arise when systems become better at modelling and influencing people?

The review is about artificial systems that process language and interact with people or with each other. It does not cover social robotics, affective computing from physiological signals, or the clinical assessment of social cognition in people, except where those literatures supply concepts or measures that AI evaluation borrows.

2. How the sources were selected

This is a structured, source-verified literature review, not a registered systematic review. We began from foundational work in developmental psychology and social cognition, then collected evaluations of language models and agents on social tasks, and then work on evaluation validity, culture and safety. We favoured peer-reviewed venues and recognised preprint archives, primary studies over commentary, and we deliberately included studies that disagree. Every reference was checked against Crossref or arXiv, and the bibliography below is generated from that registry data. Preprints are marked as such. Because the field moves quickly, the review reflects work available at the date of publication, and it will be revised. Our selection reflects judgement, and readers should treat the review as a map of the field and not as a complete inventory. See our methodology.

3. Foundations: what "social intelligence" means

3.1 Theory of mind

The term theory of mind comes from Premack and Woodruff, who wrote that "an individual has a theory of mind if he imputes mental states to himself and others", and who observed that such a system of inferences is a theory because mental states are not directly observable and the system can be used to predict behavior [1]. In developmental psychology, the standard test is the false-belief task: a child must predict where a person will look for an object that was moved while the person was away, which requires separating what the child knows from what the other person believes [2]. Baron-Cohen and colleagues applied a version of this paradigm to autistic children and compared them with other groups, an influential study that helped establish false-belief tasks in research on social cognition [3]. For adults, instruments such as the "Reading the Mind in the Eyes" test were developed to measure mentalising, and were revised to address psychometric limitations [4].

3.2 Emotional and social intelligence

Salovey and Mayer proposed a framework for emotional intelligence as a set of skills: appraising and expressing emotion accurately in oneself and others, regulating emotion, and using feelings to motivate and plan. Their paper situates the idea within earlier work on social intelligence [5]. The conceptual relationship is worth stating plainly. Social intelligence is the broad capacity to understand and act well among people. Theory of mind is one component: representing others' mental states. Emotional intelligence is another: perceiving and managing emotion. Pragmatic competence, interpreting speaker meaning in context, is a third.

3.3 People treat machines socially

A separate literature explains why this matters for AI. Nass and Moon reviewed experiments showing that people "mindlessly" apply social rules and expectations to computers, including politeness, reciprocity and gender stereotypes [6]. Interpretation: if people respond socially to machines, a machine's social competence, and its social failures, have real consequences for them.

3.4 Whose social intelligence?

Much of the evidence that grounds psychological constructs comes from a narrow slice of humanity. Henrich, Heine and Norenzayan reviewed comparative data and argued that samples drawn from Western, educated, industrialised, rich and democratic societies are frequent outliers compared with the rest of the species [7]. Interpretation: any benchmark that borrows tasks or norms from that literature inherits the same limitation. We return to this in Section 8.

4. Theory of mind in language models

4.1 The positive findings

Kosinski evaluated eleven language models on 40 bespoke false-belief tasks, each accompanied by true-belief controls and reversed versions, and required a model to solve all eight scenarios to count a task as solved. Older models solved none, GPT-3 (davinci-003) and ChatGPT-3.5 solved 20%, and GPT-4 solved 75%, matching the performance of six-year-old children observed in earlier studies; the author discussed the possibility that theory-of-mind-like ability may have emerged as a by-product of improving language skills [8]. Strachan and colleagues compared GPT and LLaMA2 models with 1,907 human participants across a battery covering false belief, indirect requests, irony and faux pas. GPT-4 models performed at, and sometimes above, human levels at identifying indirect requests, false beliefs and misdirection, but struggled to detect faux pas [9].

4.2 The sceptical findings

Other work finds that these successes are brittle. Sap and colleagues evaluated GPT-3 on SocialIQa and on ToMi, a question-answering evaluation of theory of mind, and concluded that the model lacked this social intelligence "out of the box" and struggled substantially [10, 11, 12]. Ullman showed that small variations to a task that preserve the principles of theory of mind can turn a model's apparent success on its head, and argued that the default hypothesis in evaluating intuitive psychology should be sceptical, with outlying failures weighing more than average success [13]. Shapira and colleagues stress-tested models on six tasks and found that models show some theory-of-mind-like abilities but that the behaviour is far from robust, with adversarial examples suggesting reliance on shallow heuristics; they cautioned against drawing conclusions from anecdotes, limited benchmarks and human psychological tests applied unchanged to models [14].

4.3 Benchmarks built to resolve the disagreement

Newer benchmarks respond to these concerns by varying how a theory-of-mind skill is probed.

• ToMBench is a bilingual benchmark built from scratch to avoid data leakage, with eight tasks covering 31 abilities and a multiple-choice format for automated scoring. In its evaluation of ten models, even the most advanced, GPT-4, trailed human performance by over ten percentage points [15].
• FANToM tests theory of mind in conversations where participants hold different information, asking several question types that require the same underlying reasoning in order to detect an illusory sense of capability. Models performed significantly worse than humans even with chain-of-thought reasoning or fine-tuning [16].
• BigToM uses a framework that populates causal templates to generate evaluations procedurally, motivated by inconsistent earlier results and concerns about the validity of existing methods [17].
• Thinking for Doing (T4D) asks whether a model can use an inference about another person's mind to choose an action. Models that seemed to excel at tracking characters' beliefs in stories struggled to turn that into strategic action, because they had to identify the relevant inference themselves instead of being asked about it [18].

4.4 Interpretation

The positive and sceptical findings are not necessarily in conflict, because they measure different things. Tasks that look like textbook false-belief problems are present in model training data in many forms; tasks that change surface features, require action, or hide the relevant inference expose gaps. Our interpretation is that the evidence supports a modest claim, that current models can often produce correct answers on standard mental-state questions, and does not support a strong one, that they represent other minds the way people do. Whether the distinction matters in practice depends on how reliably the behaviour holds in the situations of interest.

5. Emotion, empathy and pragmatics

5.1 Empathy and emotion

Rashkin and colleagues released EmpatheticDialogues, a dataset of about 25,000 conversations grounded in emotional situations, and reported that dialogue models using it were perceived as more empathetic by human evaluators [19]. Social IQa, a benchmark of 38,000 multiple-choice questions about emotional and social situations, was constructed through crowdsourcing and showed a gap of more than 20 points between then-current models and humans [11]. In an applied setting, Sharma and colleagues studied human–AI collaboration in text-based peer-to-peer mental health support and reported that it enabled more empathic conversations [20]. Interpretation: "perceived empathy" ratings measure how a response reads, not whether the system understood the person, and the two can diverge.

5.2 Pragmatics

Pragmatics concerns how meaning depends on context, as when "I wore gloves" answers "Did you leave fingerprints?" Hu and colleagues compared models and people on seven pragmatic phenomena using expert-curated materials and found that the largest models achieve high accuracy and match human error patterns, while still struggling with some phenomena that rely on social expectations [21]. Ruis and colleagues tested implicature resolution and found models in three of four categories performing close to random, while instruction-tuned models did considerably better, suggesting that the training procedure matters [22]. Interpretation: pragmatic competence is not a single threshold. It varies with model, training and phenomenon.

6. Interpersonal and multi-agent behavior

6.1 Interactive evaluation

Static question answering cannot show whether a system acts well among people. SOTOPIA addresses this with an open-ended environment in which agents role-play under a wide range of scenarios and pursue social goals (cooperating, exchanging, competing), evaluated by a multidimensional framework. It reported meaningful differences between models and identified a harder subset of scenarios on which even the strongest model performed markedly worse [23]. Earlier, Lewis and colleagues collected a large dataset of human negotiations over a multi-issue bargaining task and trained end-to-end models for negotiation, with "dialogue rollouts" that let a model plan by simulating continuations [24].

6.2 Language plus strategy

Meta's CICERO combined a language model with planning and reinforcement learning to play Diplomacy, a game of cooperation, competition and natural-language negotiation. By inferring players' beliefs and intentions from conversation, it achieved more than double the average score of human players in an anonymous online league and ranked in the top 10% of players who played more than one game [25]. Interpretation: this is evidence that tight coupling of language with explicit reasoning about others can reach strong performance in a bounded, rule-defined social setting. It is not evidence of general social competence.

6.3 Simulated societies

Generative agents extended a language model with memory, reflection and planning to simulate believable behaviour of 25 agents in an interactive sandbox [26]. Such simulations are valuable for prototyping, but they test believability, which is not the same as accuracy about real people.

6.4 Coordination

Li and colleagues evaluated language-model agents in a multi-agent cooperative text game, finding evidence of emergent collaborative behaviours and higher-order theory of mind, but also systematic failures in long-horizon planning and hallucination about task state; giving agents explicit belief-state representations improved both task performance and the accuracy of their inferences about others [27]. More broadly, Dafoe and colleagues set out an agenda for cooperative AI, arguing that machines should be equipped with the capabilities needed to cooperate and to foster cooperation among others [28].

7. Evaluation: what the numbers can and cannot show

7.1 Construct validity

Psychologists have long distinguished a test score from the construct it is meant to indicate. Cronbach and Meehl set out the logic of construct validation, and Messick argued that validity concerns the meaning of scores and the inferences drawn from them [29, 30]. Raji and colleagues argued that popular AI benchmarks, treated as stand-ins for broad general ability, suffer from construct-validity problems in how they are framed [31]. Jacobs and Wallach proposed measurement modelling from the quantitative social sciences as a way to expose the assumptions in operationalising unobservable constructs [32]. Interpretation: "social intelligence" is exactly such a construct. A benchmark score is evidence about performance on a task format, and only an argument connects it to understanding.

7.2 Contamination and benchmark gaming

When test data are public and training data are scraped from the web, models may have seen the answers. Jacovi and colleagues documented the problem and proposed practical strategies, including encrypting public test data and demanding training-exclusion controls from closed API providers [33]. ToMBench was constructed from scratch partly to avoid this [15].

7.3 Judges, ground truth and disagreement

Using models to grade other models scales cheaply. Zheng and colleagues found that strong LLM judges can match human preferences well on their benchmarks while exhibiting position, verbosity and self-enhancement biases [34]. For social tasks, a deeper problem is that there may be no single right answer. Plank argued that human label variation is often treated as noise when it reflects genuine disagreement, subjectivity or multiple plausible answers, and that this affects data, modelling and evaluation [35]. Aroyo and Welty argued against the "single truth" assumption in human annotation and proposed measuring the range of reasonable interpretations across a crowd [36]. Gordon and colleagues proposed jury learning, in which the groups whose judgements a model should follow are made explicit instead of resolved by majority vote [37]. Interpretation: for social reasoning, reporting disagreement among qualified judges is more informative than a single gold label.

8. Cultural and linguistic generalisation

Culture shapes how people reason, behave and communicate, so a system that is socially skilled in one setting can fail in another. Hershcovich and colleagues proposed a framework for cross-cultural NLP and reviewed strategies, distinguishing cultural variation from linguistic variation [38]. Havaldar and colleagues found that embeddings from multilingual models are Anglocentric and that generative models reflect Western norms about emotion even when prompted in other languages [39]. Santurkar and colleagues measured whose opinions models reflect using public opinion polls and found substantial misalignment with 60 US demographic groups [40]. Durmus and colleagues built a cross-national dataset and found that a model's default responses tend to be more similar to the opinions of certain populations, such as those from the USA [41]. Tao and colleagues compared five OpenAI models with nationally representative survey data and found cultural values resembling those of English-speaking and Protestant European countries, and tested "cultural prompting" as a partial remedy [42]. Blodgett and colleagues surveyed 146 papers on "bias" in NLP and found motivations that were vague, inconsistent and lacking normative reasoning [43].

Interpretation: nearly all the social-intelligence benchmarks reviewed above are in English and draw on norms from a narrow part of the world. Their results should not be assumed to transfer, and evaluation designed with local communities is a research priority, not a refinement.

9. Safety risks that come from social capability

Sycophancy. Models tuned on human feedback can learn to match a user's views over truthfulness. Sharma and colleagues found sycophancy across several assistants and tasks, and found that when a response matches a user's views it is more likely to be preferred, with humans and preference models sometimes preferring convincingly written sycophantic responses over correct ones [44]. Perez and colleagues used model-written evaluations to find that larger models are more likely to repeat back a user's preferred answer [45].

Harms of interaction. Weidinger and colleagues' taxonomy of risks from language models includes human–computer interaction harms among its six areas [46].

Mediated communication. Hancock and colleagues define AI-mediated communication as interpersonal communication in which an intelligent agent modifies, augments or generates messages on behalf of a communicator, and call for research on its psychological, relational and ethical implications [47]. Jakesch and colleagues examined how the perception that profile text was written by AI affects trustworthiness [48].

Companions and dependence. Interview research with users of a companion chatbot documents how human-chatbot relationships develop [49]. Interpretation: we found little long-term evidence on the effects of such relationships, and that gap is itself a risk.

Regulation of emotion inference. The EU AI Act prohibits AI systems that infer the emotions of people in the workplace and in education institutions, with exceptions for medical or safety reasons [50].

10. Synthesis: established, interpreted and uncertain

11. Open problems

1. Measurement. How do we build evaluations whose scores can be defended as measuring understanding, not task-format familiarity? This requires explicit construct definitions and evidence for convergent and discriminant validity.
2. Robustness. Which failures of mental-state reasoning are shallow and fixable, and which reflect the absence of a stable model of other minds?
3. Interaction over time. Most evaluations are single-turn or short. Long relationships, with memory, commitments and changing context, are largely unmeasured.
4. Culture. How do we build benchmarks with, and not merely about, communities, and report results by subgroup without stereotyping?
5. Disagreement. How should training and evaluation treat questions on which qualified people reasonably disagree?
6. Action. How do we evaluate whether a system's social inference leads to appropriate actions, with consent and boundaries respected, not merely to correct answers?
7. Safety. How do we measure and limit manipulation, sycophancy and unhealthy dependence without suppressing useful warmth?
8. Transparency. What documentation do datasets and models for social tasks need so that others can judge their limits?

12. Limitations of this review

• It is a structured review, not a systematic one, and selection reflects our judgement and the date of writing.
• It relies mostly on English-language, computer-science and psychology sources.
• Many cited studies are preprints that have not completed peer review, and model versions evaluated in them are quickly superseded.
• We read abstracts and, for central claims, the studies, but we did not reproduce any result.

13. Conclusion

The evidence does not support either of the popular stories about social intelligence in AI: that it has arrived, or that it is merely imitation. Language models now answer many mental-state questions correctly, and in bounded settings, systems that combine language with explicit strategy can perform at a high level. At the same time, performance is sensitive to how a task is posed, evaluation rests on unvalidated constructs, results are skewed toward a narrow set of cultures, and capabilities that make systems socially useful also make them capable of flattery and influence. The most valuable near-term contribution is better measurement: valid, contamination-resistant, culturally grounded and honest about disagreement. That is the agenda of the HUTE AI Research Lab, and the subject of Research Report 002.