How does the First Amendment apply to ‘deepfakes’?

This is part of a weekly series on AI and free speech.

If you follow politics, you’ve probably seen an ad made with AI — and if you live in an area with a contested midterm race, you’ve probably seen several. 

The Republican primary in Kentucky’s 4th congressional district featured an ad depicting the incumbent, Thomas Massie, in an AI-generated throuple with congressional progressives, while an ad from Massie allies showed his rival Ed Gallrein abandoning President Trump in a hail of bullets. “Trump was in the foxhole,” the narrator says. “Woke Eddie Gallrein tucked his tail and ran.” 

In Georgia, gubernatorial underdog candidate Brad Raffensperger created an AI caricature of his opponents dueling with pugil sticks to mock their ugly brawl to lead the polls — a battle that has itself been waged with AI ads. 

And in Texas, the most controversial ad of the cycle put Senate candidate James Talarico in a dress, singing an AI-crafted rendition of The Sound of Music’s “My Favorite Things,” with repurposed lyrics to have Talarico singing that some of his favorite things include “changing the gender of all your offspring.”

The Talarico ad drew condemnation from across the political spectrum, part of a larger backlash against the use of AI to create unauthorized, highly realistic depictions of people in an unflattering light. These concerns have fueled a wide array of legislation aimed at curbing their spread. 

Is training AI on copyrighted materials protected by the First Amendment?

A wave of lawsuits risks giving copyright holders a veto over a transformative new medium of expression.

Read More

At the milder end are disclosure regimes like Kentucky’s, an example of the approach most states have taken: The law lets a candidate sue to block an ad that lacks a “clear and conspicuous” AI disclaimer, like the throuple ad above. At the harsher end are criminal bans. Texas passed the nation’s first in 2019, criminalizing deepfake videos published within 30 days of an election with intent to influence the election. Minnesota’s version reaches back 90 days, and a 2024 amendment added a remarkable penalty: A candidate convicted of sharing a deepfake forfeits their nomination or office.

Some proposals reach broader than elections. The NO FAKES Act, which recently advanced out of the Senate Judiciary Committee with unanimous support, would offer a right to seek damages over an unauthorized digital replica of one’s likeness. The bill’s “one-pager” summary points to some of the biggest examples that have inspired worry about deepfakes: an AI-generated version of Tom Hanks used for “advertisements for a dental plan that he never appeared in,” for one example, and a principal at a Baltimore, Maryland, high school “framed as a racist by an AI-generated recording of his voice,” for another. The bill promises to cover both. 

But proposals targeting deepfakes also sweep in the protected use of AI depictions for news, commentary, and, notably, parodies of public figures and elected officials. And at core, that’s what the depiction of Massie in a throuple, the pugil dueling Georgia candidates, and Talarico in a dress are — parodies recalling classic themes and motifs of political cartoons and lampoons going back to the American founding. This is expression at the apex of First Amendment protection. And as courts have started sorting through anti-deepfake laws — with provisions being ruled unconstitutional in Texas, California, and Hawaii — they’ve reached the same conclusion, deeming anti-deepfake laws to be threats to traditionally protected expression in a new form. Those early returns should give legislatures pause and encourage them to contemplate the scope of what they’re restricting. 

An old tradition 

In November 1968, The Los Angeles Times published a cartoon by Paul Conrad caricaturing then-Los Angeles Mayor Sam Yorty’s reported desire to be newly elected Richard Nixon’s secretary of defense. The cartoon depicts an ignorant Yorty eagerly awaiting what he presumes is his imminent escort to the White House, but is actually a group of orderlies with a straitjacket. 

Yorty sued the newspaper, claiming the quotation attached made readers think Yorty was claiming he had been appointed secretary of defense, and, generally, arguing that the cartoon made readers believe he “was insane and should be placed in a straight jacket.” Yorty’s claim was easily dismissed. As the California appellate court emphasized on appeal: “No reasonable person would interpret the cartoon as a report that Mayor Yorty had actually made the statement shown in the caption or that he was in fact mentally deranged or insane.” 

The court rightfully pointed to a long and cherished tradition in human society of depicting adversaries saying and doing things they never said or did. This extends from primitive ridicule unearthed in artifacts from antiquity to political cartoons that date to Benjamin Franklin’s savaging of the British empire at the time of the American founding. Perhaps most famous is Thomas Nast’s brilliant mockery of New York’s corrupt Tammany Hall political machine during the 19th century, depicting Boss Tweed as a vulture feeding on the city, often with incriminating captions under his likeness.

The fabricated quotations and disparaging depictions were the lifeblood of these cartoons, but they, along with parody and satire broadly, still received the highest degree of protection under the First Amendment. This was decisively affirmed in 1988 when the Supreme Court protected a Hustler parody interview depicting Jerry Falwell in a drunken tryst with his own mother in an outhouse — elaborated upon through words Falwell obviously never spoke. The Court grounded the unanimous holding in the proud tradition preceding it: “graphic depictions and satirical cartoons have played a prominent role in public and political debate … our political discourse would have been considerably poorer without them.”

A good portion of the AI-generated content currently generating controversy — including many political ads — fits comfortably within this tradition. They place a recognizable public figure in an exaggerated or implausible scenario to ridicule a perceived trait, position, or failing. The fabricated image and words are used as a vehicle to emphasize an underlying political criticism. That is the heartland territory of satire (the use of exaggeration, irony, or invented scenarios to engage in political or social critique) and parody (imitation directed at the person, work, or style being imitated, usually for purposes of criticism or ridicule).

But they look real!

But, you might ask, aren’t the fake quotes and actions in an AI-generated “deepfake” delivered through a photorealistic depiction of a person? An AI depiction of a political opponent is inherently more believable than, say, a cartoon, which is clearly a fake depiction. Shouldn’t that change the analysis?

This argument runs into a few problems. First, realism has never been a disqualifying factor for satire to receive protection. If you look at the Falwell parody, for example, his quotes are presented in what at face value looks like a perfectly realistic sponsored interview, formatted like the real ad campaign it spoofed, with a real photo of Falwell to boot. 

The Supreme Court held the fake ad was still unquestionably protected by the First Amendment. That’s because readers are let in on the fabrication by numerous pieces of context revealing the satirical intent and nature of the content. The level of realism is just one lens to evaluate.

That’s important. In the cartoon context, the cartoons alone don’t imply a quotation is faked — plenty of caricatures have been paired with real quotations. Thomas Nast’s satirical intent was revealed by the cartoon medium in addition to the exaggerated characters, distinct symbols, and political themes. For the Falwell parody, the obvious tell is the absurdity of the language.

So too does the AI-assisted Talarico ad leave plenty of contextual tells revealing its satirical nature: the cartoonish dress, the outrageous lyrics, the over-the-top delivery. And for a reasonable Kentuckian, it’s obvious that Massie is not in an intimate relationship with members of the squad and that his opponent Ed Gallrein has not actually been in a war with Trump. Again, those ads are just the latest iterations in an American tradition of disparaging depictions using satire to persuade people politically, only enhanced by the capabilities of new technology. 

But it’d be a mistake to view deepfakes as novel — or the issues raised by them insurmountable. Audiences have been recalibrating to realistic fabrications for nearly as long as there have been images to fabricate. This also extends to the election context. In 1950, Maryland Senator Millard Tydings was defeated after a tabloid circulated a spliced photograph placing him in friendly conversation with Communist Party leader Earl Browder — a meeting faked with scissors and paste. In recent decades, Photoshop has put that capability in digital form on every desktop. It’s produced famous deceptions and parodies, like John Kerry’s fabricated rally with Jane Fonda that caused a stir in the 2004 election, the doctored image of Hillary Clinton and Osama bin Laden shaking hands submitted as a joke to a 2007 photoshop contest, and the 2008 depiction of Obama goofily holding a phone upside down. 

While many have feared the effects such digital manipulation would have on society, politics, and our shared sense of reality, people have largely adapted by adopting “that’s photoshopped” as a reflex to any image that intuitively feels not quite right.

The counter is often this: Some people are just really dense. We have to restrict deepfakes for the sake of the idiots among us.

But those fears are even older than Photoshop. Boss Tweed, the story goes, didn’t fear editorials attacking him, as he believed his constituents were illiterate — but he did worry they could be affected by “them damn pictures.” Today’s political commentators now fear an imagined prototypical cable news-watching uncle uncritically believing Talarico is a dedicated theater kid with a penchant for songs about transgender children. 

The problem with that is the evidence that deepfakes are making a massive dent in elections is sparse.

What effects are deepfakes having on elections? 

Heading into 2024 — a year of national elections across the globe — fears abounded that AI-generated misinformation could be democracy’s breaking point. The World Economic Forum ranked misinformation and disinformation as “the most severe short-term global risk the world faces,” singling out AI as the key amplifying factor. 

When we got through the year, two sets of researchers combed through the use of AI and the broader literature around it to examine its effects: Princeton’s Sayash Kapoor and Arvind Narayanan; and Oxford’s Felix M. Simon and University of Zurich’s Sacha Altay.

Kapoor and Narayanan analyzed every instance of election-related AI content in WIRED’s AI Elections Project, a database that tracked 78 cases of AI use in elections worldwide. The researchers found much of it fairly innocuous — improving campaign materials, translating languages, and anonymous AI avatars used to avoid retribution from authoritarian governments.

And for uses that were deceptive, Kapoor and Narayanan estimated what the same content would have cost without AI, including hiring a Photoshop expert or a video editor. In every case, the answer was “no more than a few hundred dollars.” 

Can the government require ID before you use artificial intelligence?

Age checks for AI could become ID checks, threatening privacy, anonymity, and the freedom to ask questions, learn, and explore ideas online.

Read More

It shouldn’t be surprising, then, that the researchers did not find a massive migration of misinformation to AI formats. The News Literacy Project, for example, documented known 2024 U.S. election misinformation and found “cheap fakes” — miscaptioned photos, jump-cut edits, slowed audio — used seven times more often than AI-generated content. Abroad, the gap ran wider: an Indian fact-checker reviewed an order of magnitude more cheap fakes than deepfakes, and in Bangladesh cheap fakes were more than twenty times as prevalent. Looking at the entire global picture, Meta reported that under one percent of the misinformation its fact-checking partners reviewed worldwide was AI-generated. 

So why haven’t deepfakes swallowed the universe of misinformation? Simon and Altay pointed to a University of Vienna paper from 2021, which found that videos derive their evidentiary power not from quality or content but from their source: “An audience may find even the most realistic video evidence unconvincing when it is delivered by a dubious source. At the same time, an audience may find even weak video evidence compelling so long as it is delivered by a trusted source.” In their view, political content is often persuasive because of a trust relationship between source and consumer. And while AI has made it more convenient for trusted sources to produce and share fabricated content, production costs are not a bottleneck for them — it’s already plenty cheap enough to find, share, and create non-AI fabrications with conventional tools. 

Outside those trusted sources, the researchers found strong limits to public persuasion — noting findings on the limited boosts to sales from consumer advertising and the minimal impact of repeated exposure to persuasive political content. As they summarized, “research on the limited influence of mass persuasion clashes with the widespread assumption that people are gullible and easily persuaded.” They hypothesize that what is really driving worries about AI-generated content is what they call a “third-person effect,” the fact that we simply perceive “others to be much more susceptible to negative media effects than ourselves.” 

The value of AI-generated deepfakes

So if we have reason to believe the impact of deepfakes on political persuasion are limited, why are we seeing politicians use them in the examples we reviewed?

Well, they’re funny. Like political cartoons, people use AI depictions to turn criticism into a joke — using imitation, exaggeration, and ridicule to grab attention for a political point. Courts confronting deepfake regulation have recognized as much, placing AI-driven satire within the “long-held American tradition of ridiculing and criticizing candidates and elected officials.” 

But AI’s photorealism does add a certain quality — an effectiveness, even — that isn’t inherent to previous media in this tradition. And I’m not referring to the believability of the underlying “false” depiction. What’s enhanced is the joke. The medium opens up new possibilities for comedy and ridicule through the absurdity of the photorealism. Take this example of an AI-generated video mocking President Trump by depicting him on a romantic escape with Vladimir Putin. It’s funny to people because the public figures look real — even if everyone knows they’re not. The same content would simply have a different appeal and different audience in, say, a hand-drawn medium. That’s because the fact of the depiction so closely resembling the target of the criticism adds a new dimension to the satire that makes otherwise uninteresting content engaging and entertaining, and the ease of using AI tools allows more people to create and engage in the critique facilitated by the medium.

This makes regulation that burdens or restricts the medium especially concerning. As the Yorty court said, political cartoons were particularly important because they were a unique medium for getting across certain critiques and ideas that can’t be made in quite the same way in other mediums. They “communicate in graphic form a statement of editorial opinion which might otherwise require paragraphs of written material to express.” AI’s photorealism similarly offers new opportunities for comedy and critique of politicians and public figures that will gradually reveal themselves as the medium develops — that is, if it’s allowed to develop without landing satirists or campaigns in legal trouble. 

Now, I don’t want to pretend the ad putting Talarico in a dress is the Mona Lisa. Admittedly, much of AI-generated content will strike readers as pretty bottom-of-the-barrel — “AI slop,” as it has come to be termed. But as the federal district court in Kohls v. Bonta emphasized, “novel mediums of speech and even low-brow humor have equal entitlement to First Amendment protection.” Whether the content provides genuine social value or not is up to viewers and creators, not the government.

The law already has answers

But what about egregious, plainly deceptive uses of AI-generated content that lack social value and do real damage? A notable example proponents of deepfake legislation point to is an AI-generated Joe Biden robocall from 2024 that encouraged New Hampshire Democrats not to vote in the state’s presidential primary. 

In many cases, this kind of deception will already be barred under existing law. In the election context, we have laws governing impersonation of a candidate and voter suppression. For reputational harm, certain false or deceptive speech that causes specific, targeted harm to individuals is already punishable under narrowly defined First Amendment exceptions. If, for example, someone creates and distributes a deepfake that is intended to and actually does deceive others into thinking someone did something they didn’t do, the depicted individual could have a claim for defamation or false light. 

Existing law’s capabilities in this area aren’t hypothetical. When the perpetrator behind the Biden robocall was revealed to be a Democratic operative hoping to expose the need for new regulation of AI and deepfakes, he was given a wake-up call in the form of a steep federal fine and multiple criminal charges under existing legal frameworks. That kind of threat is taken care of under existing law.

Lawmakers might be tempted to meet us “halfway” by simply requiring a visible disclosure for AI-generated content. But disclosure has issues, too. California’s AB 2839 exempted satire and parody from its restrictions so long as the AI-generated content carried a disclaimer meeting the statute’s formatting requirements — text that, on a phone screen, could swallow the video it labeled. The Kohls court saw the flaw plainly: “a mandatory disclaimer for parody or satire would kill the joke.” Satire works by letting the audience take in the joke on their own terms — through the cartoonish imagery, absurd dialogue, ridiculous premises. A label announcing the fabrication in advance destroys part of what has long endeared audiences to satire. These risks are compounded by the fact that, as FIRE previously warned in comments to the FCC, an AI label “is likely to be interpreted by the public as an admission that the political ad is likely to be false or misleading.”

But again, neither restrictions nor disclosures appear necessary. The current research presents little evidence that the fears about AI-generated content are materializing, or that the medium is proving more formidable at persuasion than the long line of content manipulation tools which preceded it. In fact, the lesson from those tools has been that society is fairly resilient to novel fabrications. And for the rare fabrication that is truly dangerous or damaging, the law already has existing remedies. For the rest — the fake throuples, the imaginary pugil sticks, the digital show tunes — it’s time for lawmakers to get in on the joke.