At the end of January 2024, X was inundated with graphic and fake photographs of Taylor Swift. While celebrities have long been victims of photo leaks and cyberattacks, this time was different because they were generated thanks to artificial intelligence.
The photographs were temporarily reported through fans of the “Shake It Off” singer and deleted after circulating on the author’s profile for less than a day. However, it took enough time for them to go viral despite the platform’s policies against non-advertising. -Consensual nudity. A report by disinformation research firm Graphika later revealed that the photographs were created on 4chan, where users encouraged each other to generate deepfakes with sexual connotations in an effort to circumvent content policies surrounding nudity through female celebrities.
Unfortunately, Swift’s experience is rarely unique. Marvel actress Xochitl Gomez, who is only 17 at the time of writing, said on The Squeeze podcast that she fought to remove X’s deepfakes and shared the intellectual effect on it. had on it. Gomez and Swift are just two of countless women who have recently fallen victim to deepfakes that depict them in sexual ways.
“People have used the media to smear people, but that hasn’t replaced it. What it has replaced is how available they are now,” Siwei Lyu, a professor of computer science at the University at Buffalo, told The Daily Beast.
Late last year, AI imaging platform CivitAI became popular thanks to its “Bounties” feature, which encouraged users to create deepfakes in exchange for virtual rewards. Nearly all of the bonuses created went to women, according to data from 404 Media. It included women who were neither celebrities nor public figures, but personal citizens.
Experts expect the situation to get worse, especially as more and more online incel communities use those technologies. Henry Ajder, an expert advisor on artificial intelligence and deepfake, told The Daily Beast that this has been a developing challenge for years and that CivitAI is an example of a platform strongly tied to this kind of evolution.
He said CivitAI has “a hotbed of not only artistically created content, but also erotic content. “It is an express position to find express wisdom and other people have started using it for pornographic content.
Ajder also describes the platform’s generation as “agnostic or dual-use,” saying that once there, it can be used in any way, “while others are explicitly designed to create pornographic content without consent. “within the incel culture through platforms like Reddit and 4chan.
“The threshold is very low,” Hera Husain, founder of Chayn, a nonprofit that helps victims of gender-based violence and trauma, told The Daily Beast. “It’s an easily accessible approach that allows other people to satisfy the darker fantasies they could possibly have. [. . . ] They would possibly feel that there are no victims, but it has massive consequences for those other people.
It’s not just deepfakes that have penetrated incel culture. There are even studies that show that AI brides will make incels even more dangerous. With this generation allowing them to shape and their perceptions of the so-called “ideal woman,” there is a threat that it will increase perceptions of genuine women. When they realize they can’t do it or when a woman is unapproachable, as in the case of Swift or Gomez, incels launch deepfake campaigns. At least incels can force those women to do it. Whatever they want.
“Governments are just playing catch-up; generation is faster than its ability to regulate,” Belinda Barnet, a professor of media at Swinburne University, told The Daily Beast.
This becomes even more damaging when we look at global contexts. Patriarchal norms in other countries occasionally put women at greater threat when they become victims of such campaigns. In more conservative countries, even the forgery of a woman may be enough to force her circle of relatives to banish her or, in extreme cases, use violence against her. For example, in late 2023, an 18-year-old woman was killed by her father because of a photo of her with a man that police suspected had been tampered with.
It doesn’t matter if the symbol is fake. The fact that their symbol is related to such a representation is enough for society to exclude them. “It’s as much about believing that the symbols are genuine as it is about pure malice. It’s another kind of revenge trauma,” Ajder explained.
As AI generation becomes more accessible, it also becomes a less complicated barrier to entry for global incels who would have arguably faced language barriers. In South Asia, where Husain concentrates much of his work, it is also more difficult to fight the radicalization of incels. , either socially or politically. ” They don’t have as strong a resistance to radicalization as they see in the incel community,” he explained.
Lyu argues that policies related to freedom of expression and access to generation vary around the world and therefore may have other impacts. “In the U. S. , employing AI generation teams to create content. . . It’s freedom of speech, but other people can enjoy it too. “it. Drawing that line becomes very difficult. Although in China there are very strong limitations on the use of this generation, this is imaginable but prevents positive uses of the same generation line.
Incel culture existed long before AI generation teams became popular. Now that they have become widespread, it may not be long before those communities adopt them to cause more harm and trauma. The challenge will definitely get worse before it gets better.
“In terms of incel culture, this is a weapon in their twisted arsenal to abuse women, perpetuate stereotypes and make the twisted concepts they have about women even more visceral,” Ajder said.
Do you have a tip? Submit it to The Daily Beast here.