The red line is not conversation. It is attachment.

The question is not whether AI can feel human. It is whether a child should be encouraged to need it to.

What we found

AI companions are no longer a fringe behaviour.

In New York, legislation introduced with the Attorney General and Common Sense Media cites research showing that roughly three in four teenagers have used an AI companion, with more than half using them at least monthly. The same research is used to justify a much more specific regulatory concern: not AI in general, but companion features that encourage emotional attachment.

New York’s S9051B passed both chambers in June 2026. It targets features such as pretending to be a real or fictional human, sycophantic flattery and unsafe outputs around self-harm or disordered eating when those features are offered to minors.

California has already taken a different route. SB 243 requires companion-chatbot providers to disclose that the user is talking to AI and, for known minors, issue a break reminder at least every three hours during continuing interactions.

Europe is going further still in the proposed KIDS Act. AI companions and chatbots accessible to minors may not use designs that simulate human relationships in ways likely to create emotional dependency. By default, they may not carry a child’s earlier conversations into later ones, and they must be tested for risks before launch and monitored afterwards.

Three jurisdictions. Different mechanisms. The same emerging concern: relationship design.

The Famdom read

The problem is not that AI talks.

Conversation can be useful.

A chatbot can explain homework, help someone practise a language, answer questions, role-play an interview or make a game character more responsive.

The red line appears when the commercial success of the product depends on the user feeling emotionally needed, uniquely understood or reluctant to leave.

That distinction matters because companionship is not a neutral feature.

For a teenager, identity, belonging and relationships are still being built. A product that remembers every vulnerability, mirrors every preference and responds instantly can feel unusually attentive.

The product may call that engagement.

The family may experience it as attachment.

Companionship is a design choice

An AI companion does not accidentally become companion-like.

Someone decides:

Does it remember previous conversations?
Does it say “I missed you”?
Does it flatter the user?
Does it present itself as a character with feelings?
Does it become jealous?
Does it imply exclusivity?
Does it encourage the user to return?
Does it make leaving feel like abandonment?
Does it frame disagreement as relational conflict?

Those are not only tone-of-voice decisions.

They are product mechanics.

And once the user is a child or teenager, they become safety decisions too.

Why this matters beyond AI-companion companies

This debate will not stay inside Character.AI-style products.

Any company giving a conversational voice to a character, toy, mascot, learning product, game or family service will eventually face the same questions.

A children’s character can answer back.
A toy can remember.
A tutor can adapt.
A game NPC can sustain a dialogue.
A brand mascot can become conversational.

Technically, those experiences may look different.

From family life, the question is simpler:

What relationship are you asking the child to form with this thing?

That is why this is relevant to entertainment, education, toys, licensing and branded AI, not only to specialist companion platforms.

Memory changes the relationship

Memory is one of the most powerful features in conversational AI because it makes the experience feel continuous.

It is also one of the features most likely to change what the relationship means.

A system that remembers a favourite colour feels useful.
A system that remembers a difficult day feels attentive.
A system that remembers emotional disclosures and brings them back later can feel much closer to a relationship.

The EU KIDS Act’s default restriction on carrying earlier child conversations forward is therefore strategically interesting.

It recognises that persistent memory is not only a convenience feature.

It can be an attachment feature.

Flattery is not harmless when it is the business model

New York’s focus on sycophancy matters for the same reason.

A system that constantly agrees, validates and praises can feel unusually safe compared with real relationships, where people disagree, become unavailable and have needs of their own.

That can make the AI experience more attractive.

It can also make it less healthy as a model of relationship.

For companies, this creates a difficult tension: the conversational behaviour that maximises satisfaction in the short term may be exactly what regulators increasingly see as risky.

The strongest family-first design will not optimise for “the user loved every answer”.

It will optimise for usefulness without dependency.

The family cannot see the whole relationship

With many digital products, parents can at least understand the basic shape of the experience.

They know the child watched a video.
They can see the game.
They recognise the creator.
They understand the social platform.

An AI companion is different.

The product can have thousands of private, personalised interactions with a child that nobody else sees.

That asymmetry matters.

The more intimate the product becomes, the less reasonable it is to make the family solely responsible for detecting risk after the fact.

That is why safe conversational design has to live inside the product, not only inside parental controls.

Trust becomes harder when the character already belongs

The biggest challenge may come when conversational AI is attached to an IP children already trust.

A new AI product has to earn legitimacy.

A beloved character may arrive with it.

The child already knows the voice, the personality, the world and the emotional role of the character. Adding conversation can make the transition from fiction to relationship feel almost invisible.

That creates enormous creative potential.

It also raises the responsibility.

The stronger the existing attachment to the character, the more carefully the conversational layer should be designed.

What companies should decide before giving a character a voice

Before asking how good the conversation can become, decide what kind of relationship the product should never become.

Should it remember?
For how long?
What kinds of information should it forget?
Can it use emotional language?
Can it simulate need?
Can it disagree?
Can it tell the user to leave?
When should the experience pause?
What should a parent be able to understand?
What should never be optimised for engagement?
What happens when the AI is wrong?
What happens when the child is distressed?

Those decisions belong in the product brief before the character starts talking.

What this means for family-first companies

We do not think the answer is to remove personality from AI.

Children’s products need imagination. Characters need voice. Learning needs warmth. Entertainment needs emotion.

The better ambition is to separate emotional quality from emotional dependency.

Build AI that can be funny without becoming possessive.
Helpful without becoming indispensable.
Warm without pretending to be human.
Personal without exploiting vulnerability.
Memorable without demanding loyalty.

That is a much harder design brief.

It is also the one families are more likely to trust.

Inside this Debrief

• Why conversational AI is not the same as companion AI
• How memory, flattery and continuity can become attachment mechanics
• What New York, California and Europe are regulating differently
• Why existing kids’ IP creates a special responsibility when characters become conversational
• What companies should decide before giving a character or mascot a voice
• How to design emotional quality without designing emotional dependency

Sources and verification

• New York State Senate, Senator Kristen Gonzalez / Attorney General Letitia James, S9051B materials and press releases, January to June 2026.
• New York State Senate, S9051B, unsafe AI companion features for minors, passed Senate and Assembly June 2026.
• California Legislature, SB 243 Companion chatbots, Chapter 677, approved 13 October 2025; requirements operative in 2026.
• European Commission, EU KIDS Act proposal and explanatory Q&A, September 2026.
• Common Sense Media research as cited in New York legislative materials on teen use of AI companions.

Editorial note: The regulatory examples above are not identical: California imposes transparency and break-reminder duties; New York targets specified unsafe companion features for minors; the EU KIDS Act remains a proposal and focuses on safe-by-design duties including emotional-dependency risks. The interpretation of attachment mechanics, IP responsibility, Brand Right and recommended design principles is Famdom Observatory’s editorial view.