California’s new chatbot law has a free speech problem

Fifteen years ago, the Supreme Court in Brown v. Entertainment Merchants Association struck down a California law barring the sale of violent video games to minors. The Court held that the state’s legitimate power to protect children does not include “free-floating power to restrict the ideas to which children may be exposed.” It also rejected the idea that the interactivity of video games made them legally different from books, plays, or films.

Today, California is again overreaching to block access to an interactive medium in the name of child safety. Its recently enacted “Adam’s Law” (Senate Bill 1119) restricts what “companion chatbots” can say to users suspected of being minors and pressures companies to verify users’ ages. The law’s main requirements take effect on July 1, 2027.

Adam’s Law will require people who make chatbots available to users in California to take “reasonable measures” to prevent certain chatbot communications with minors, such as encouraging suicide or drug use. But the law also regulates other kinds of conversation. A chatbot cannot claim to be human, encourage a minor to conceal chatbot use, discourage a minor from sharing health or safety concerns with appropriate people, claim to have a “special or unique understanding” of the minor based on a purported special relationship, or use “excessive praise or flattery” disproportionate to the context.

The California legislature has decided that government officials, not the American people, get to decide how powerful new communications tools can be designed and how we’re allowed to use them.

But what counts as excessive praise? How much understanding is special or unique? Toymakers can be held responsible for choking hazards or defective parts. But there’s a clear difference between regulating physical hazards and suppressing speech. 

The problem isn’t lawmakers trying to keep children safe. The bill requires developers to try to determine what speech crosses these vague lines and take measures to suppress speech otherwise fully protected by the First Amendment. Apply the same standards to books, film, or ordinary conversation, and the government interference with free speech becomes obvious. SB 1119 is no different: it’s a clear example of government policing expression.

The content rules are only one part of the law. Operators must also censor their models to try to limit sprawling and vaguely defined potential risks to minors, including things like emotional harm, specifically “severe and reasonably foreseeable psychological or emotional harm to a reasonable child.” Chatbots must be automatically set to parent-locked default settings that disable a chatbot from remembering conversations, disable push notifications, and limit use to two hours in any single day and only one hour at a time. Operators must provide parental controls, new disclosures, and crisis-response rules. Government officials will be able to go after violators and fine them, and individuals will be able to sue.

Why the First Amendment applies

The First Amendment comes into play when the government restricts the sharing of information. Expression does not lose First Amendment protection simply because a person used AI to help create it. As FIRE argued in Garcia v. Character Technologies, developers’ expressive choices in producing desired outputs — whether through selecting training data, fine-tuning the model, system prompts, or post-inference filters — are protected by the First Amendment.

Your speech, their rules: How foreign governments censor Americans

From arrests to online censorship, foreign governments are increasingly targeting Americans for speech protected by the First Amendment at home.

Read More

If chatbot outputs weren’t speech under the First Amendment, the government would have considerable latitude to regulate their content. That risks allowing the ideological preferences of political officials to dictate the ideas and information people receive through this technology. The stakes are especially high given that almost half of all American adults use them. 

Minors have First Amendment rights, too. Protecting minors is an important goal, but that interest — as the Supreme Court in Brown made clear — is not a blank check for the state to determine what lawful ideas and information they are allowed to receive.

Moreover, usage caps, memory defaults, notification settings, disclosure mandates, parental controls, and audits might look like product regulations. But their effect on expression is alarming. Disclosure mandates may impermissibly compel speech, forcing developers to say things they otherwise would not say. Design requirements replace the editorial choices of AI developers with government mandates. A rule requiring a conversation to end after an hour puts a hard limit on how long someone can engage in lawful expression. A rule disabling conversational memory hobbles the information available to the user in chatbot interactions.

The bottom line is that the California legislature has decided that government officials, not the American people, get to decide how powerful new communications tools can be designed and how we’re allowed to use them.

Who’s asking?

Regulating chatbot conversations with minors requires companies with chatbots to know everyone’s age or treat everyone like a child.

Adam’s Law presents chatbot operators with a choice about determining whether the person typing is under 18. Operators can rely on a standardized age-range indicator under California’s Digital Age Assurance Act or extend the law’s child restrictions to everyone. Under that act, operating systems are required to collect a user’s age at account setup.

FIRE has warned that age assurance can involve identity verification, especially where the methods include providing a government ID. This puts private data at risk. People are increasingly sharing deeply personal reflections with artificial intelligence that were once reserved for diaries and journals. Imagine a teen recounting her parents fighting at dinner, a college student coping with long-term illness, or a young athlete tracking weight. Those conversations are sensitive and private. But California’s law means anonymous access to these conversations will be harder.

The alternative is even worse. Applying the law’s restrictive elements to everyone means the default rules would govern adults’ interactions as well. Adults would have to live with two-hour caps a day, no continuity in their conversations, and — puzzlingly — settings only a parent can change.

The Supreme Court warned against child-proofing speech. In the words of one decision, it’s like the state burning the house to roast the pig.