Deepfakes Archives

FBI sets up 24/7 National Election Command Post to handle security threats

November 5, 2024

The FBI has established a 24/7 National Election Command Post in Washington, D.C., to address any threats that may arise before and after Election Day. This weeklong effort brings together over a dozen federal agencies and roughly 80 personnel per shift to monitor for federal election crimes, cyber threats, foreign interference and domestic terrorism risks.

The FBI’s Election Command Post is based in the Strategic Information and Operations Center at FBI headquarters. Agents and officials are prepared to respond to threats as results come in.

Agencies supporting this operation include the Department of Justice, Department of Homeland Security, U.S. Capitol Police and the U.S. Secret Service, among others.

Each of the FBI’s 55 field offices are also operating local command posts. The offices are working closely with state and local election officials to pass on relevant tips and complaints.

James Barnacle is the deputy assistant director of the Criminal Investigative Division of the FBI. He said they’re prepared for the election.

“We feel like we’re well positioned to handle and triage information that comes in. We have the people in place,” Barnacle said.

The NECP’s mission is to protect the election process from potential disruptions while ensuring that information flows swiftly between federal, state and local partners.

This multiagency effort will continue through at least Nov. 9, handling all significant complaints and any incidents that could impact election integrity.

Foreign countries influencing 2024 election, Microsoft says

October 23, 2024

With less than two weeks until the 2024 U.S. presidential election, foreign adversaries are increasing their efforts to disrupt the democratic process, Microsoft warns. Russia, Iran and China are targeting key political campaigns and voters using cyber tactics and disinformation, according to a report from the Microsoft Threat Analysis Center.

Russia has concentrated its influence operations on the Harris-Walz campaign, according to Microsoft. Russian operatives have created AI-generated deepfake videos aimed at discrediting Vice President Kamala Harris.

One video falsely depicts her insulting former President Donald Trump, while another accuses her of illegal activities in Zambia.

The United States has approximately 700,000 cybersecurity job openings. Congress hopes to entice more people to enter the field.

Disinformation targeting Democratic vice-presidential nominee Tim Walz has also gained traction. One video has received more than 5 million views on social media.

Iranian hackers, tracked by Microsoft as “Cotton Sandstorm,” have been probing U.S. election-related websites and media outlets.

Despite ongoing tensions in the Middle East, Iranian operatives are preparing for cyber-influence operations targeting U.S. audiences. This is similar to their 2020 operation posing as the right-wing group Proud Boys. This time, Iran is encouraging voters to boycott the election, claiming U.S. support for Israel undermines candidates.

Leading up to the Jan. 6 Capitol riot, the FBI had as many as eight informants inside the Proud Boys Organization.

China, meanwhile, is focusing its disinformation efforts on down-ballot Republican candidates who advocate for anti-China policies. Sens. Marsha Blackburn and Marco Rubio, along with Rep. Barry Moore, have been targeted with accusations of corruption and amplified antisemitic messaging. Though these efforts have not seen high engagement, they represent China’s sustained attempts to influence U.S. political discourse.

Microsoft reports that all three countries are using AI to spread disinformation rapidly. As Election Day approaches, these foreign interference operations are expected to increase, undermine voter confidence and cast doubt on the integrity of the results. Early detection and fact-checking are critical to counter these efforts, Microsoft says.

TikTok approves 50% of political ads containing false information: Study

October 18, 2024

Social media platforms TikTok and Facebook are under scrutiny for failing to detect harmful disinformation advertisements ahead of the U.S. presidential election. A new investigation by Global Witness reveals that these platforms are still allowing misleading content to slip through their moderation systems.

In a report, Global Witness tested the election integrity commitments of TikTok, Facebook and YouTube by submitting eight advertisements containing false election claims.

Reuters

The ads were deliberately designed to breach the platforms’ policies and were submitted using “algospeak,” a technique that employs numbers and symbols to bypass content moderation filters.

TikTok approved 50% of the submitted ads. Facebook accepted one ad with harmful disinformation, an improvement from its previous performance during the 2022 midterm elections. YouTube initially approved 50% of the ads but later blocked all of them until formal identification was submitted, a more robust barrier against disinformation compared to the other platforms.

Although TikTok has improved from approving 90% of submitted disinformation ads in 2022, its ongoing issues are particularly notable given its strict policy against political content.

The platform, already under scrutiny due to potential foreign interference, faces further examination for its role in U.S. political discourse.

TikTok acknowledged that the ads submitted by Global Witness breached its advertising policies and stated that its machine moderation system had erroneously approved them.

The platform indicated that it would use the findings to retrain its moderation system. Meta, the parent company of Facebook, did not respond to requests for comment.

The Babylon Bee sues California over satire and ‘deepfake’ law

September 30, 2024

The Babylon Bee has filed a lawsuit against the state of California, challenging new laws signed by Gov. Gavin Newsom, D, that regulate satire and parody through “deepfake” restrictions. The lawsuit claims the laws infringe on free speech by requiring social media platforms to monitor and report deceptive content, which could include political satire.

In July 2024, Gov. Newsom tweeted that a parody video of Vice President Kamala Harris should be illegal, leading to legislation prohibiting such content. The new laws focus on AI-generated disinformation but also mandate labeling or removal of satire that isn’t clearly marked.

An artificial intelligence advisor for the United Nations says deepfake technology usage in elections is "the greatest threat to democracy." — The Associated Press

Babylon Bee CEO Seth Dillon argued that forcing disclaimers ruins the humor of satire and could result in penalties if not followed.

The lawsuit, supported by Alliance Defending Freedom, asserts that satire is protected under the First Amendment, which guarantees the right to political speech. The suit adds to a growing list of conservative media outlets alleging censorship.

Last year, The Daily Wire and The Federalist sued the Biden administration over similar concerns. This particular legal showdown could set new precedents for political satire in the digital age.

Newsom vetoes controversial California AI safety bill

September 30, 2024

Gov. Gavin Newsom vetoed a landmark California bill that would have established the nation’s first safety regulations for artificial intelligence. The proposed legislation sought to require AI companies to test their systems, publicly disclose safety measures and provide whistleblower protections.

Proponents of the bill argued that it was a necessary step to address the risks AI poses to infrastructure and public safety, with potential threats ranging from manipulating electric grids to creating chemical weapons.

However, Newsom opposed the bill, stating its strict requirements could hinder innovation by imposing regulations even on low-risk AI systems.

California is home to 32 of the world’s top 50 AI companies.

Newsom expressed concern that the legislation could drive developers out of the state. Instead, he announced a partnership with AI industry leaders to create more flexible safety guidelines.

California Gov. Newsom is pushing a new oil refinery mandate to address high gas prices by requiring oil companies to keep extra fuel on hand — Getty Images

While the veto is seen as a win for tech companies, experts warn that it leaves rapidly advancing AI systems unregulated.

Supporters of the bill, including Elon Musk’s X and AI firm Anthropic, argued that it could have introduced much-needed transparency in an industry that remains largely unregulated.

Critics of the bill feared the regulations might discourage investment and development, particularly in open-source software. Despite the veto, similar efforts are already inspiring lawmakers in other states to explore AI safety measures.

California continues to lead AI development, with the state already working to combat election deepfakes, protect Hollywood workers from unauthorized likeness use and prevent AI-driven discrimination in hiring practices.

Meanwhile, the Biden administration has proposed an AI Bill of Rights, but federal legislation regulating the rapidly growing industry has yet to be introduced.

Senate Foreign Relations Chair Cardin targeted in deepfake attack

September 26, 2024

The chairman of the Senate Foreign Relations Committee was the target of a deepfake attack in which someone posed as a top Ukrainian official during a Zoom call. The incident, which happened earlier this month, involved someone impersonating the now-former Ukrainian Foreign Minister Dmytro Kuleba during a conference with Sen. Ben Cardin, D-Md.

The story was exclusively reported by Punchbowl News. Straight Arrow News reached out to the committee to confirm the incident and the details.

The Punchbowl report said the person impersonating Kuleba asked questions participants thought were bizarre like, “Do you support long-range missiles into Russian territory?”

According to Punchbowl, the Senate Security Office warned a group of leadership aides and security chiefs to be vigilant about others trying to do the same.

Sources told the outlet that Kuleba’s voice was likely recreated using artificial intelligence. The security notice stated the deepfake had “technical sophistication and believability.”

The motive is still not confirmed, however, people briefed said Russia may have done this in an attempt to create propaganda with a senior U.S. senator saying he supports attacks on Russian territory.

The FBI is investigating.

Newsom signs election ‘deepfake’ ban, critics raise free speech concerns

September 18, 2024

Gov. Gavin Newsom, D-Calif., signed a law Tuesday, Sept. 17, banning the use of AI-generated deepfakes in political campaigns, marking the nation’s most aggressive regulation of artificial intelligence in elections. The law targets deepfakes that impersonate political candidates and spread misinformation ahead of the 2024 election.

Newsom introduced the legislation after Elon Musk shared a manipulated video of Vice President Kamala Harris, which falsely depicted her as an incompetent candidate. The governor stated that deepfakes threaten the integrity of democratic elections and needed to be addressed urgently.

The law allows courts to block deceptive content and imposes civil penalties on those who distribute it. Starting next year, social media platforms will be required to remove flagged deepfakes. Additionally, political ads using AI must include disclaimers to inform the public about altered content.

Musk, a frequent critic of Newsom, attacked the law on his social media platform X. He compared Newsom to “The Joker” and argued that the legislation infringes on free speech. He accused the governor of trying to outlaw political parody and satire. Musk’s posts reignited debates about the balance between free speech and preventing misinformation.

This law comes at a pivotal time as AI-generated videos are expected to play a significant role in the 2024 presidential election. While supporters claim the law is necessary to protect voters, critics warn it could stifle free expression. As the legal debate continues, California remains at the forefront of AI regulation in politics.

Tech giants commit to removing nude images from AI training data

September 13, 2024

Several big tech companies have committed to removing nude images from AI training datasets. The news is in response to the Biden administration’s call for action.

Companies including Adobe, Anthropic, Cohere, Microsoft, and OpenAI have pledged to implement what they call responsible sourcing practices and incorporate feedback loops to guard against the output of image-based sexual abuse.

Tech companies have pledged to remove nude images from AI training datasets “when appropriate and depending on the purpose of the model.”

The White House reports an alarming increase in image-based sexual abuse. Platforms have become breeding grounds for the creation, dissemination, and monetization of deepfakes, alongside a concerning rise in sextortion cases.

The 2023-24 school year witnessed a global epidemic of deepfake incidents, with teenage girls being the primary targets of explicit content created and shared by their peers.

Commitments from these AI companies specifically target the prevention of nonconsensual intimate images of adults and child sexual abuse material.

However, critics argue that voluntary efforts are insufficient to tackle the complex challenges posed by AI-generated sexual content. The president of the Cyber Civil Rights Initiative, Mary Franks, emphasized that responsible and accountable practices by tech companies could have prevented the current crisis.

Another group of tech giants, including Bumble, Discord, Match Group, Meta, and TikTok, also announced a set of voluntary principles aimed at preventing image-based sexual abuse.

Deepfake abuse to generate nude photos of kids on the rise

August 30, 2024

A newly released report by Thorn, a tech company that works to combat the spread of child sex abuse material, showed deepfake abuse is becoming all too common. This is partially due to the rise of artifical intelligence.

Thorn said that is because “undressing” apps are widely available and inexpensive. Individuals can also use other user-friendly, easily-accessible, AI-powered tools to create deepfake versions of nude photographs that look believable.

In a recent survey of more than a thousand students ages 9 to 17-years-old, Thorn found 11% of them had friends or classmates who used AI to generate deepfake nudes of classmates. Another 10% of students refused to answer the question.

Thorn_23_YouthMonitoring_Report Download

The survey also found that almost 1 in 4 kids (24%) aged 13 to 17-years-old said they had been sent or shown an actual nude photo or video of a classmate or peer without that person’s knowledge. Only 7% of the students surveyed admitted that they had personally shared a nude photo or video without the other person’s knowledge.

However, sharing real nudes is a fairly large problem among teens. Almost 1 in 3 (31%) teenagers surveyed said that is normal for people their age. Only 17% of them admitted to sharing nudes of themselves, however.

Thorn noted this trend coincides with a rise in sextortion among young people. About 6% of those surveyed said someone had blackmailed them by threatening to share their nude photos unless they sent money or did something else the blackmailer asked.

New lawsuit filed over AI-powered deepfake nude websites

August 19, 2024

San Francisco filed a lawsuit against 16 AI-powered websites, marking the first government action to combat non-consensual deepfake pornography. The case aims to shut down these websites, which allow users to create AI-generated nude images without consent.

The sites collectively received more than 200 million visits in the first half of 2024.

Users can upload photos of fully clothed individuals, including children, which are then manipulated by AI to create non-consensual nude images.

Individuals used the technology to create deepfake nudes of people ranging from singer Taylor Swift to Rep. Alexandria Ocasio-Cortez, and everyday citizens like middle-school girls.

In Beverly Hills, California, school administrators expelled five eighth-graders for creating and sharing AI-generated nude images of 16 female classmates.

Victims of these AI-generated nudes often face severe psychological trauma, damaged reputations, and in some cases, suicidal thoughts.

The pervasive nature of online content makes it exceptionally challenging for victims to trace the origin of their manipulated images.

“This is an enormous national and global problem,” Yvonne Mere, San Francisco’s chief deputy attorney, said. “Businesses and websites that conduct these malicious practices have evaded enforcement. Individual victims of this conduct have little practical and legal recourse. It’s very difficult for victims to determine what websites are used to nudify their images because these images don’t have any unique or identifying marks that links you back to the websites.”

The lawsuit, filed on behalf of Californians, alleges violations of state and federal laws against deepfake and revenge pornography and child exploitation.

It seeks to permanently restrain those operating the websites from creating deepfake pornography in the future and assess civil penalties.

FBI sets up 24/7 National Election Command Post to handle security threats

Foreign countries influencing 2024 election, Microsoft says

The Babylon Bee sues California over satire and ‘deepfake’ law

Newsom vetoes controversial California AI safety bill

Senate Foreign Relations Chair Cardin targeted in deepfake attack

Newsom signs election ‘deepfake’ ban, critics raise free speech concerns

Tech giants commit to removing nude images from AI training data

Deepfake abuse to generate nude photos of kids on the rise

New lawsuit filed over AI-powered deepfake nude websites

Certified unbiased reporting

Transparent and credible

Newsletter

Stay connected