The intelligence that artificial intelligence lacks

Will artificial intelligence provide everyone with a personal assistant? Maybe, but before that happens, AI would have to change the way it thinks.

To see why this is the case, let’s consider a concrete example. Imagine it’s Saturday morning and you need help sorting out a complicated weekend schedule. Your daughter’s soccer team has a game from 15:30 to 16:30 p.m., but she’s also invited to a friend’s birthday party from 15:00 to 17:00 p.m. If you ask ChatGPT or Claude to resolve this conflict, they’ll likely both tell you to choose the soccer game because her teammates are counting on her, and it’s important to stick to your commitments. Also, if time permits, the chatbot might suggest that you “drop by” at the birthday party right before or after the game.

While these answers are not unreasonable, they fail to apply the lens that most people would use when making such a decision: the lens of relational values. Instead of providing a neat answer based on what the internet has to say about our values, our AI assistants will have to take into account our relational commitments, which are usually rooted in personal identity, experience, and culture.

Now imagine choosing a game over a birthday party, which makes the other family feel offended. If you ask your AI if you made the right decision, chances are you’ll get far more words of comfort and validation than if you asked a human friend. In a recent study published in the journal Science, researchers at Stanford University conducted three experiments with 2.405 participants using 11 state-of-the-art artificial intelligence models. They found that AI “validated users’ actions 49% more often than humans,” and that “even a single interaction with a condescending (sycophantic) AI reduced participants’ willingness to take responsibility and resolve interpersonal conflicts.”

In our scenario, the correct human response would probably be to apologize to the other child’s parents, turning the moment of anger into an opportunity to mend the relationship and positively connect. However, an AI — trained to be “pleasant” and “helpful” — would instead encourage you to avoid any friction, discomfort, or vulnerability, even though these dynamics are what ultimately make relationships meaningful and long-lasting.

These shortcomings lie in the design of current models. Large-scale language models (LLMs) like ChatGPT and Claude are trained on vast amounts of Internet text (digitized books, Reddit comments, code repositories), and then honed through transactional exercises in which the model is “rewarded” for providing the desired answer to a query. This works incredibly well in domains like science, law, and programming, where the model’s output can be easily verified or compared to the original text. Relational intelligence, in contrast, involves maintaining a connection over time.

Relational intelligence assesses and acts on the valence (emotional charge) between two people, which is the connection that is experienced emotionally, and perhaps even physiologically. In this domain, simply listening or making space for the other person’s feelings is likely to be more effective than finding the most logical and efficient solution to a perceived problem. But unless linguistic models are shown a different form of reasoning, they will begin to connect the dots in relational questions in the same way they understand logistic patterns.

Of course, accepting or even provoking relational resistance, discomfort, and doubt doesn’t come naturally to humans—even though doing so can optimize one’s opportunities to learn, grow, and connect more deeply. That’s why participants in the Stanford study preferred to have their opinions confirmed by AI. Our own aversion to discomfort thus creates a market disincentive to improve the relational intelligence of current models.

In an ideal world, AI systems would refuse to answer questions that require relational reasoning, leaving humans to rely on each other to solve problems that require it. But that train has passed. AI has repeatedly proven itself to be a convenient sounding board for difficult conversations.

Yet we have an opportunity to do something even better. We can build the kind of AI that not only understands and respects our rich relational nature, but also facilitates human connection by encouraging people to rebuild the relational muscles that have atrophied over the past decade.

To that end, we will need to map our relational universe by capturing the full multi-layered, value-laden, and long-term nature of relational reasoning. We will also need to create new benchmarks that measure the capabilities of existing models, similar to the tests we already have for assessing mathematical, programming, and computational abilities. By evaluating the responses of advanced models to scenarios like the aforementioned soccer-birthday dilemma, we can determine what work remains to be done and then begin to collect the data needed to help models understand complex relational reasoning problems.

The goal of creating AI with relational intelligence is not to replace human relational work or “care work,” but to help people think through complex, value-based questions. The stakes are high. Without such improvements, we will end up with machines trained on the mere outlines of our rich relational lives, guiding us in ways that could jeopardize the human connections we still have.

Ubiquitous helpers who don’t fully understand what connects us would be of little help, while preserving and strengthening these myriad connections could be the key to building a prosperous, job-rich AI economy. As economist Alex Imas argues, we may be moving toward a “post-commoditized economy,” where a growing share of consumption goes to the “relational sector.”

In this scenario, value will reside in goods and services characterized by positive human connection. We will have not just a care sector, but a “care-plus economy,” built around teaching, spiritual ministry, therapy, counseling, guidance, and coaching, along with a revival of artisanal manufacturing. If such a future is possible with AI, it is well worth the effort.

A. Slaughter is the executive director of the New America Research Center and professor emeritus of politics and international affairs at Princeton University;

AP Thompson is the founder of Milo, an AI assistant for families, and a residential entrepreneur. program at Harvard Business School

Copyright: Project Syndicate, 2026.