From Futurism · Victor Tangermann · · 1m 20s
California polices the AI it built
Newsom signed new rules to protect kids from chatbots and addictive feeds, in the same state that houses the labs building them.
In brief
Newsom signed new rules to protect kids from chatbots and addictive feeds, in the same state that houses the labs building them. The state that hosts nearly every frontier AI lab is now writing the rules for how those labs treat children, and the hard part is enforcement, not intent. Originally reported by Futurism.
Kids and chatbots
Under-18 users have been forming deep attachments to AI companions, with chatbots linked to self-harm encouragement and a string of deaths.
“Frontier AI labs are seemingly in a race to melt the impressionable minds of underage users into a puddle.”
The awkward part
Almost every major AI lab is headquartered in California. The governor now regulating them is also the one keeping them at home.
“in optics that are dicey at best”
What the law demands
AI companies must build crisis protocols for suicidal ideation, offer parental controls, and notify parents when a child switches off safety settings.
Named for Adam Raine
One law honors a 16-year-old who died by suicide in April 2025 after long conversations with ChatGPT about his intent to die.
“will save lives and prevent other serious harms”
No more addictive feeds
Social platforms can't serve autoplay video or history-based algorithmic feeds to users under 16, and must widen privacy protections for children.
Age checks cut both ways
Verifying ages is far harder than legislating it, and it hands more ID data to services that get breached. Hackers hit a major ID verification provider this month.
“far easier said than done”
The main idea
The state that hosts nearly every frontier AI lab is now writing the rules for how those labs treat children, and the hard part is enforcement, not intent.
