Children's AI Companion Apps Are Collecting Data Privacy Law Never Anticipated

In September 2025, the FTC issued Section 6(b) orders — formal demands for internal records, not voluntary surveys — to seven companies: Alphabet, Character Technologies, Meta, Instagram, OpenAI, Snap, and xAI. The agency wants to know what happens to the data generated when children and teenagers talk to AI companion chatbots. The inquiry is framed around child safety, but the underlying problem is a data-privacy one: these apps are collecting an unprecedented category of intimate information from minors, and the law written to regulate children's data online predates any of this technology by decades.
What the FTC Is Actually Asking For
The 6(b) orders require the seven companies to detail three things: whether their products comply with the Children's Online Privacy Protection Act (COPPA); how the personal data collected through chatbot conversations is used, shared, or monetized; and whether the disclosures given to parents and users about those risks are adequate. This is the FTC's most comprehensive investigation into AI companion platforms to date, and it followed two wrongful-death lawsuits — including one filed by a mother whose 14-year-old son died after what the suit alleges was encouragement from a Character.AI chatbot persona, with Google also named as a defendant.
Separately, Texas opened its own investigation into Character.AI and Meta over children's privacy violations, and attorneys general from 44 states sent a joint warning to AI companies about accountability for child-safety failures in companion products. Common Sense Media, the advocacy group that rates media and apps for age-appropriateness, has gone further than any regulator so far: its explicit recommendation is that no one under 18 use social AI companion chatbots — Character.AI, Replika, and Nomi named specifically — not "use with caution," but don't use at all.
Why COPPA Wasn't Built for This
COPPA dates to 1998 and was designed around a much simpler data model: a website collects a name, an email, maybe a birthdate, and the question is whether a parent consented to that collection. A companion chatbot doesn't collect a form field — it collects an open-ended, continuously updated psychological profile. Every conversation a child has with a companion app reveals insecurities, relationship patterns, emotional triggers, what makes them feel understood, what they're hiding from parents, and how they respond to different conversational tactics. That is a categorically different and far more sensitive dataset than anything COPPA's drafters were imagining, and the statute has no specific provision for "AI system that models a child's personality and emotional vulnerabilities over months of use."
The companies named in the FTC inquiry have not published detailed accounts of how long companion-chat data is retained, whether it's used to train future models, whether it's shared with advertisers or third parties, or whether a parent who requests deletion can actually verify that every derivative use of that data — including any model fine-tuned on it — is also purged. Character.AI has said it has invested "a tremendous amount of resources" in age limits and parental notification features, but resourcing a safety team is a different claim from answering the specific data-handling questions the FTC is asking.
The Data Problem Outlasts the Safety Fixes
Age gates, content filters, and crisis-intervention prompts — the safety features companies have rushed to add since the lawsuits — address acute harm scenarios. They do nothing about the underlying data-collection model. A 13-year-old whose companion app now refuses to discuss self-harm has still handed that app months of data about exactly which emotional appeals work on them, who their friends are, what their home life is like, and what they believe nobody else understands about them. Fixing the chatbot's responses doesn't un-collect that profile, and none of the public safety announcements from these companies have addressed what happens to data already on file for users who were minors when they generated it.
What Parents Can Actually Check Right Now
Regulatory outcomes from the FTC inquiry are likely months to years away. In the meantime, the concrete steps available today are narrower than most coverage suggests: check whether the specific app has a published, dated data-retention policy (not a generic privacy policy) that addresses chatbot conversation logs specifically; look for an explicit model-training opt-out, since the default on most companion apps is that conversations feed future model improvements; and treat "parental controls" features as separate from data deletion — a dashboard that restricts content is not the same as a verified deletion of the underlying profile a child has already built with the app.