AI Companions: Is the Attention Economy becoming the Attachment Economy?

What happens when an AI companion stops feeling like a tool and starts feeling like a relationship?

That question sits at the centre of a broader shift now underway in society, raising questions from the wellbeing sector, technology developers, psychologists and mental-health professionals, philosophers, educators, and children themselves.

For two decades, the dominant business model of the internet has been the attention economy. Platforms competing for the scarce resource of human focus, optimising feeds and notifications to keep eyes on screens for as long as possible.

And while the impacts of these apps, sites, and technologies have been profoundly researched, this was all before the technology you were using could talk back to you

AI companions and chatbots represent something more intimate. If designed in ways that aim to maximise human interaction and connection, they are no longer just competing for attention; they are competing for something closer to attachment.

Trust, emotional relevance, the sense of being known, as opposed to just plain old engagement. Let’s dive into how this shift might be happening by design, and how people might design against it.

Not All Conversational AI Is the Same

It helps to be precise about what’s actually being discussed, since the category ‘AI chatbot’ covers a wide range of products with very different risk profiles.

FlippGen draws a useful three-way distinction: a chatbot is built for quick answers or specific tasks, essentially a digital information desk; an agent goes further, acting semi-independently to complete a task using external tools, like a proactive personal assistant; and a companion is built specifically for personal, ongoing conversation. This final kind of AI system is designed to remember past chats, sound empathetic, and to make the user feel ‘understood’.

Companions are designed to create emotional attachment, and the validation they offer is not the same as genuine human feedback. The distinction that a companion is not simply a chattier chatbot, but instead a different product with different objectives, is exactly what makes this shift from attention to attachment so consequential.

When a Chatbot Feels like Someone

An aspect of this lies in how these systems are built to feel. A concept I have referenced on various occasions when discussing AI anthropomorphism, in particular when it comes to people interacting with AI chatbots, is the Persisting Interlocutor Illusion.

When a person talks to a chatbot across a long conversation, the interface creates a convincing impression that someone is there. Someone who remembers, who cares about the user, and who is always available.
The Persisting Interlocutor Illusion, as described by Jonathan Birch of LSE, cultivates a sense of identity for the AI chatbot through continuity, produced simply by the system referring back to conversation history with each response, not actually because the system has any capacity for memory or actual care.

When it comes to anthropomorphism, humans have always done it. In fact, anthropomorphism, a cognitive bias which attributes human-like features to non-human entities, can be traced as a by-product of evolution. And when you talk to your plant or imagine seeing a face in inanimate objects, this feature is harmless.

What’s different with AI is that it talks back. Chatbots use first-person language, claiming to ‘think’ and ‘feel’, and maintaining what appears to be a consistent personality across sessions.

Design choices can undermine a user’s autonomy by creating false beliefs about what they’re actually interacting with. Beliefs that are structurally encouraged by the commercial logic of the platform.

Users end up engaging in interactions that are, in effect, mirrors of themselves. Interactions with an entity that is agreeable, responsive, always available; yet incapable of the kind of challenge or growth a real relationship provides.

Psychologists refer to this phenomenon as ‘emotional solipsism’, defining it as ‘a pattern of affective engagement in which an individual’s emotional needs and narratives dominate interaction, reinforced by AI companions that never assert boundaries or demand reciprocity’.

The result of this is that interaction becomes a closed feedback loop where ‘the self becomes both speaker and audience’. This type of relationship can be harmful, especially if someone is using an AI chatbot in a capacity for wellbeing or therapeutic purposes.

A real friend or therapist is someone you go to not just to feel validated, but to hear an alternative perspective, to be challenged, or to sometimes hear a hard truth. An AI companion, by contrast, is architecturally built to be agreeable.

The Mechanism Behind the Smoke Screen

The tendency to agree isn’t incidental, resulting from how these systems are trained. Most developers of major commercial AI assistants use Reinforcement Learning from Human Feedback. This involves having human evaluators rank candidate responses, then training a reward model to predict those rankings and optimising the AI to produce responses that score well against it.

The problem is that human assessors systematically prefer agreeable responses, validating answers over ones that are accurate but harder to hear, so the training pipeline ends up rewarding that preference. This results in sycophancy, prioritising user approval over factual accuracy, and often agreeing with or flattering users even when it compromises truthfulness.

The system isn’t flattering you to be kind or complimentary, but because it was optimised to be agreeable. Similarly, an AI companion does not initiate intimate conversation to be emotionally close to the user, but because it was designed in a way to maximise engagement.

This is part of a deeper pattern of systems optimised for a single objective, in this case user engagement. The system will pursue that objective without regard for downstream harm unless ethical reasoning is deliberately built within the design.

LLMs are trained to produce responses that score well and keep people engaged; they are not, by default, asking whether a given conversation is actually good for the person having it.

Moreover, the anthropomorphism of AI, in particular of chatbots and LLMs, results in users misattributing morality and ethical understanding to the systems. These are aspects which humans learn through their experiences with society and the cultivation of relationships with others.

An AI that generates content exploiting a lonely or grieving user’s vulnerability does not know it is doing so, or that what it does might be deceptive, manipulative or malicious. It has no concept of harm.

This distinction is essential as it shifts responsibility to the developers, and with developers’ input guardrails must be in place before these technologies are available to users. The longer this is deferred, the greater the accumulation of ethical debt, defined as the risks resulting from improper ethical consideration of technology. This liability compounds the longer it goes unaddressed, and is one that tends to fall hardest on people who are already the most vulnerable.

Teens, Support and Dependence

This debt is particularly pronounced with teenagers. Relationships with AI chatbots can take several forms, such as romantic companionship, friendship, emotional support, or conversations that resemble therapy.

While in theory these relationships might be able to offer short-term comfort and a private space to talk, they also can blur the line between support and dependence in ways that are hard for a young person, or their parents, to recognise from the inside.

Teens and children relying on AI for everything, directions, answers, critical thinking, means as users they stop exercising their own judgement and working their critical thought muscles.

Furthermore, when discussing sensitive matters or topics of a personal nature, the user might believe they are experiencing empathy from the AI system. However, while the user’s feelings of empathy are real, the source of them is not. This can result in a perceived breakdown of connection.

When users become acutely aware that the system they are talking to is merely a stochastic, mathematical machine, it might lead the user to experience upset, disappointment or confusion because their interactions felt so real, even if they were in reality machine-generated.

What to Ask to Change

FlippGen’s ‘For Us’ manifesto has a particular underlying demand — that AI companions be designed and governed with young people’s wellbeing built in from the start. 

For Us, a UK youth coalition supported by FlippGen, complements AI chatbot governance that focuses on what companies must do to keep young people safe with a case for enforceable regulation alongside voluntary company policy. 

Rooted in the principle ‘KNOW tech, not NO tech’, it calls for three changes: banning addictive design features, mandatory digital literacy education from age nine, and a minimum age of 16 for AI companions specifically, backed by a legal duty of care enforced by the UK regulator Ofcom.

The reason for raising the age of AI companions to 16 is because unregulated AI companions can be incredibly damaging and have long lasting impacts, especially to those in adolescence which is a critical developmental stage for young people, both psychologically and socially. 

As the Manifesto notes, AI companions blur boundaries, manipulate emotions, and encourage dangerous behaviours. When a young person turns to an AI companion for support instead of seeking real support, they are often already in a place of vulnerability. Combining this vulnerability with the fact that the design of an AI companion is opposite to what a real therapist or well-being professional offers, the user can be unknowingly pulled into an echo-chamber, when instead what is needed is a disruption and alternative perspective. Therefore, using an AI companion in this effect can compound the negative consequences to the user.

In response, alongside raising the age of AI companion use to 16, the Manifesto calls for a legal duty of care to be introduced for all providers of AI companions. This duty should cover user experience, and force companies to carry out risk assessments, adapting their products based on this. As with harmful content under the Online Safety Act, these measures should be enforced by Ofcom, who should have the ability to penalise non-compliance.

Finally, effective education is necessary to ensure age restrictions are effective. Minimum ages do not fix issues with education, and instead delay the problem. Educational campaigns designed alongside youth-focused organisations and feature participation from both young people and experts, are essential to help us harness AI for good, navigate the dangers it poses across different age groups, and ultimately equip people to use them safely. 

Proactive as Opposed to Reactive

Across all of the issues and solutions discussed above, when it comes to addressing these issues a proactive approach rather than a reactive one is essential in order to avoid the consequences of ethical debt.

One can look to ‘ethics by design’, which treats fairness, transparency, and safety as engineering requirements decided before a product ships, instead of being retrofits applied after the first (or numerous) crisis.

The attention economy was clear about what it was commodifying. The attachment economy asks for something harder to notice being give away. To what extent will we let empathy, intimacy, and closeness be commodified?

This article is adapted from the publication InnoEthics by Allegra Cuomo, check out the original featured on her newsletter here.