San Francisco filed a lawsuit against 16 AI-powered websites, marking the first government action to combat non-consensual deepfake pornography. The case aims to shut down these websites, which allow users to create AI-generated nude images without consent.
The sites collectively received more than 200 million visits in the first half of 2024.
Users can upload photos of fully clothed individuals, including children, which are then manipulated by AI to create non-consensual nude images.
Individuals used the technology to create deepfake nudes of people ranging from singer Taylor Swift to Rep. Alexandria Ocasio-Cortez, and everyday citizens like middle-school girls.
In Beverly Hills, California, school administrators expelled five eighth-graders for creating and sharing AI-generated nude images of 16 female classmates.
Victims of these AI-generated nudes often face severe psychological trauma, damaged reputations, and in some cases, suicidal thoughts.
The pervasive nature of online content makes it exceptionally challenging for victims to trace the origin of their manipulated images.
“This is an enormous national and global problem,” Yvonne Mere, San Francisco’s chief deputy attorney, said. “Businesses and websites that conduct these malicious practices have evaded enforcement. Individual victims of this conduct have little practical and legal recourse. It’s very difficult for victims to determine what websites are used to nudify their images because these images don’t have any unique or identifying marks that links you back to the websites.”
The lawsuit, filed on behalf of Californians, alleges violations of state and federal laws against deepfake and revenge pornography and child exploitation.
It seeks to permanently restrain those operating the websites from creating deepfake pornography in the future and assess civil penalties.
