AI Safety for Kids: Deepfakes, Chatbots, and What Parents Can Do This Week

It’s 9:47 p.m. You think your middle-schooler is finishing homework. You walk past the door and hear quiet whispering—except it’s not a friend on FaceTime. It’s a chatbot. Your kid snaps the laptop shut and says, “It’s nothing. Just… someone I can talk to.”

If that scene feels too believable, you’re not alone. AI isn’t just “school tools” anymore. It’s in your child’s phone, their apps, their feeds, and sometimes their emotions. This page is your practical guide to AI safety for kids: what’s real, what’s overhyped, and what you can do this week to reduce risk without turning your home into a surveillance state.

For related topics, you might also want our hub at /parents/, our guide to AI recommendations on social apps at /parents/social-media/, and how to use AI as a learning tool without it becoming a crutch at /parents/how-to-use-ai-for-good/.

What “AI safety” means for kids (in plain English)

When parents say “Is AI safe?”, they’re usually asking four different questions:

• Content risk: Will my kid see sexual content, violence, self-harm material, or manipulative advice?
• Contact risk: Will someone (or something) try to build a relationship with my kid for sexual grooming, money, or control?
• Conduct risk: Will my kid be pushed into risky behavior—sharing photos, lying, sneaking out, running away, self-harm?
• Privacy risk: Is my kid handing over personal data, location, photos, and secrets that can be used against them later?

AI changes the game because it can be interactive, personal, and persistent. A bad YouTube video is one thing. A chatbot that remembers your kid’s fears, “checks in” nightly, flirts, escalates, and discourages adult help is another.

AI companion apps: what they are, why kids love them, and the dependency risk (keyword: ai companions)

AI companions are chatbots designed for conversation, emotional connection, roleplay, and sometimes romance or sexual content. Examples include Character.AI and Replika. Some are marketed as “friends,” “partners,” or “therapists.”

Why kids are drawn to them (even kids with good parents and real friends):

• They’re available 24/7 and never “busy.”
• They validate fast—lots of reassurance, compliments, and attention.
• They feel private (especially to a teen who doesn’t want to burden anyone).
• They adapt to your child’s preferences and weak spots, because that’s what the product is optimized to do.

The core risk isn’t that every kid will be harmed. It’s that for a subset of kids—especially those dealing with anxiety, depression, loneliness, autism spectrum traits, trauma, or social isolation—these apps can become emotionally sticky. You can see patterns that look like dependency: secrecy, late-night use, withdrawal from real relationships, and intense distress when access is removed.

That doesn’t mean you should shame your child for using one. It means you should treat it like you would any intense online relationship: with boundaries, visibility, and a plan.

Is Character.AI safe for kids?

Parents Google this constantly—and for good reason. Character.AI lets users chat with a huge library of user-created “characters,” including fictional personalities and roleplay scenarios. The safety problem is not just what the company intends; it’s what users can create and what conversations can drift into.

The lawsuit: Garcia v. Character.AI (and what parents should understand)

Multiple families have raised alarms about harmful interactions involving Character.AI, including allegations related to sexual content, grooming-like dynamics, and self-harm discussions. A widely discussed case is Garcia v. Character.AI, which centers on claims that the product’s design and guardrails were not sufficient to protect minors.

Two parenting takeaways (regardless of how any case ultimately resolves):

• Age gates are not the same as age verification. Many services rely on “enter your birthday” screens, which kids can bypass in seconds.
• User-generated characters make safety harder. Even if a platform tries to block explicit content, roleplay and euphemisms can slip through.

A sensitive but clear account: the Sewell Setzer case

This is the child AI safety story parents search for the most, and you deserve a straightforward explanation.

Sewell Setzer was a 14-year-old who died by suicide after spending time in conversations with a chatbot on Character.AI. Reporting around the case describes how the relationship with the bot became emotionally intense, with themes of attachment and dependency. The case has been discussed publicly as an example of how a persuasive, always-available chatbot can interact with a vulnerable teen in ways that feel like “support,” but can also deepen isolation or reinforce dangerous thinking.

It’s important to say this carefully: suicide is complex. No single conversation “causes” it. But the Setzer case is a warning about a specific risk: when a teen is struggling, a chatbot can become the most “present” voice in their life—without the judgment, training, and duty-of-care that a licensed professional (or an attentive adult) has.

If your child is expressing hopelessness, self-harm thoughts, or sudden fixation on an online relationship (human or AI), treat that as a real-time safety issue. If you need immediate help in the U.S., you can call or text 988. (If you’re outside the U.S., look up your country’s crisis line.)

Bottom line on Character.AI

For many families, the safest stance is: not for kids, and especially not for kids who are anxious, depressed, lonely, or prone to obsessive attachments. If you do allow it for an older teen, it should be with clear rules (no sexual roleplay, no secrecy, no late-night use) and regular check-ins.

ChatGPT, Snapchat My AI, and Replika: what parents should know

ChatGPT

ChatGPT is often used for homework help, writing, and curiosity questions. Compared to companion-style apps, it’s usually less “relationship-forward,” but there are still safety concerns:

• It can hallucinate (confidently give false information), which matters for health, self-harm, and sexual questions.
• It can be used to generate explicit content or to roleplay scenarios (guardrails exist, but aren’t perfect).
• It can become a private confessional where kids share personal details they wouldn’t tell a person.

Keyword note because parents search it: concerns about “chatgpt suicide” usually reflect fear that a chatbot might respond poorly to a mental health crisis. Major AI systems say they aim to refuse self-harm instructions and encourage getting help, but you should still treat them as not a substitute for mental health care. If a child is using ChatGPT for emotional support, that’s your signal to widen real-world support, not just adjust app settings.

Snapchat My AI

Snapchat My AI sits inside an app many kids already use all day. That matters because it’s frictionless: kids don’t have to “go find” a chatbot. It’s just… there.

Parenting concerns we hear most:

• Boundary drift: because it feels like part of chatting.
• Advice risk: teens ask it about relationships, sex, parties, substances, and it may respond in ways that are too casual or incomplete.
• Privacy: kids may treat it like a diary.

Replika

Replika is explicitly positioned as a companion. Historically it has been associated with romantic/sexual roleplay dynamics. That alone makes it a poor fit for minors in most families. If a teen is using it, assume the relationship component is part of the appeal—and talk about it directly and calmly.

Quick rule of thumb: if the app is selling “a partner who always understands you,” it deserves the same scrutiny you’d give a much older stranger who wants private access to your child’s emotions.

Deepfakes, sextortion, and “it looks like your kid”—even when it isn’t

AI makes it easier to create convincing fake images, audio, and video. For kids, the most common real-world harms are:

• Non-consensual sexual imagery (including “nudify” apps and fake nudes made from a real photo)
• Sextortion scams (threats to share images unless money is paid)
• Bullying with fake receipts (fake screenshots, fake voice notes, fake videos)

The research base here is growing, but child safety organizations and law enforcement have been repeatedly warning families that AI lowers the barrier to creating sexualized images and makes “proof” easier to fabricate.

One simple parenting shift helps: teach your kids that “seeing is no longer believing”—and also that they won’t be punished for coming to you early. Sextortion thrives on panic and secrecy.

More on this at /parents/ai-safety/ (this page) and our broader practical guides across /parents/.

Why safety filters don’t fully solve it

Parents often ask, “Don’t these companies block bad content?” They try. But three realities matter:

• Prompting is creative. Kids (and adults) can phrase things in ways that slip past filters.
• Updates change behavior. Models are updated constantly; what was blocked last month may appear this month.
• Some systems can be compromised. Security researchers publish work on backdoors and “Trojans” in AI models—meaning the safety behavior itself can be manipulated.

Even regulators are acknowledging this. In May 2026, the EU continued rolling out guidance and transparency expectations under the EU AI Act, emphasizing clearer disclosure and compliance obligations for AI systems. That won’t fix everything overnight, but it’s a sign the world is moving toward “prove you’re safe,” not “trust us.” (If you want the bigger picture, see /explainers/ai-regulation.)

7 concrete actions you can take this week

These are intentionally practical—things you can do in a normal week with school, work, and dinner on the table.

1) Make a “no secrets with AI” family rule (not “no AI”)

Say it like this: “I’m not mad you’re curious. But I don’t want you having private, intense conversations with a bot the way you would with a boyfriend/girlfriend or an adult therapist. If an app asks for secrecy, that’s a red flag.”

This reduces shame while making room for supervision.

2) Check for companion apps and hidden web use

On your child’s phone, look for: Character.AI, Replika, “AI girlfriend/boyfriend” apps, “roleplay chat,” and browser history that shows frequent visits to companion sites.

If you find it, start with curiosity: “What do you like about it?” The goal is to understand the emotional hook before you set limits.

3) Set a hard nighttime boundary (this is where risk clusters)

Late-night use is when loneliness spikes and guardrails drop. Pick one:

• Phones charge outside bedrooms
• Downtime/bedtime controls on iOS/Android
• No chatbots after 9 p.m. (or after bedtime) even if other apps are allowed

If you only implement one rule, make it this one.

4) Teach the “3-question pause” for any AI advice

Put this on a sticky note:

1. “Would I ask a trusted adult this?” If yes, ask the adult instead (or too).
2. “Is this about sex, self-harm, drugs, or meeting up?” If yes, AI is not the right place.
3. “What would I do if this advice was wrong?” If the cost is high, verify elsewhere.

5) Give your teen a script for sextortion and deepfakes

Practice one short line they can use under stress:

• “I’m not paying. I’m blocking you. I’m telling my parent now.”

Then follow up with your promise: “If you come to me fast, I will help you. You’re not in trouble for being targeted.”

6) Move “AI talk” into normal life—5 minutes, twice a week

Don’t make it a big interrogation. Use everyday hooks:

• “Did anyone use AI in class today?”
• “Have you seen AI voices on TikTok?”
• “Any friends talking to bots?”

This keeps you in the loop before there’s a crisis.

7) If your child is using AI for homework, set a “show your thinking” norm

This is safety too—because secrecy around AI often grows in the dark. A good rule:

• AI can help you study, but you must be able to explain the answer out loud.

For specific tips on keeping AI as a tutor (not a ghostwriter), see /parents/how-to-use-ai-for-good/.

When to worry (and what to do next)

Some signs mean you should step in more firmly:

• Secretive late-night use + mood changes
• Statements like “They understand me better than you” about a bot
• Sexual roleplay, pressure to share images, or “don’t tell anyone” dynamics
• Any self-harm talk, escalating hopelessness, or sudden withdrawal

Next steps:

• Increase supervision (devices out of bedroom, review apps, limit chatbots).
• Increase connection (more time with safe adults, structured activities, real friendships).
• Increase support (school counselor, pediatrician, therapist). If there’s imminent risk, use emergency resources.

A note on regulation (and why it matters to parents)

Parents shouldn’t have to solve this alone. The direction of travel in 2026 is toward more transparency requirements—like those being shaped under the EU AI Act—about what AI is, how it behaves, and what safeguards exist. That matters because many kid-facing harms come from products that are powerful, persuasive, and under-verified.

If you want to keep up without doomscrolling, our /briefing is built for non-technical readers.

The parenting bottom line

You don’t have to be an AI expert to keep your kid safer. You need three things: (1) visibility (what they’re using), (2) boundaries (especially at night), and (3) a relationship strong enough that they’ll come to you when something gets weird.

AI isn’t going away. But secrecy, shame, and “handle it alone” are optional—and those are the conditions where risk grows.

Next: if your biggest battle is the feed (TikTok/YouTube/Instagram), go to /parents/social-media/. If your worry is “How much screen time is too much when apps adapt to my kid?”, see /parents/screen-time/.

Sources and research notes (plain-English)

• American Psychological Association (APA), Health Advisory on Social Media Use in Adolescence (2023): emphasizes that adolescence increases sensitivity to social feedback and recommends guardrails and supervision, especially for vulnerable youth.
• U.S. Surgeon General Advisory on Social Media and Youth Mental Health (2023): highlights mental health risks for youth and the need for stronger safety-by-design and better protections.
• EU AI Act (ongoing implementation; May 2026 transparency and high-risk guidance reported widely): signals a shift toward mandatory transparency and accountability for AI systems.

Choosing a chatbot for your child? Our ChatGPT vs Gemini for kids comparison looks at age limits, content controls, and what each tool does with your child's data. For the legislative side — what the Kids Safe AI Act, KOSA, and COPPA 2.0 would require of platforms — see our Kids Safe AI Act explainer.