Deepfake bots on Telegram make the job of creating fake nudes dangerously easy

Researchers have discovered a 'deepfake ecosystem' on the messaging app Telegram centered around robots that generate fake nudes on demand. Users who interact with these bots say they mostly create nudes of women they know using images taken from social media, which they then share and trade with each other on various channels. Telegram.

The investigation comes from security firm Sensity, which focuses on what it calls "visual threat intelligence", specifically the spread of deepfakes. Sensity researchers found that over 100 images were generated and shared on public channels Telegram until July 2020 (which means that the total number of images generated, including those never shared and those made since July, is much higher). Most users of these channels, about 70 percent, come from Russia and neighboring countries, Sensity explains. The edge was able to confirm that many channels studied by the company are still active.

Bots are free to use, but they generate fake nudes with watermarks or only partial nudity. Users can then pay a fee equal to pennies to completely “discover” the images. A "beginner's rate" charges users 100 rubles (about $ 1,28) to generate 100 fake nudes without watermark over a seven-day period. Sensity says that "a limited number" of the images generated by the robots show targets "which appear to be minor."

Both The edge and Sensity contacted Telegram to ask him why they allow this content on their app but haven't received an answer yet. Sensity says it has also contacted law enforcement authorities.

In a poll on one of the leading deep nude sharing channels (originally published in Russian and English), most users said they wanted to generate images of women they knew in "real life" .

The software used to generate these images is known as DeepNude. It first appeared on the web last June, but its creator shut down its website hours after receiving mainstream media coverage, saying "the likelihood of people misusing it is too high." ". However, the software continued to spread through backchannels, and Sensity says DeepNude "has since been reverse engineered and can be found in enhanced forms on open source repositories and torrent websites." . It is now used to power robots Telegram, which automatically handle payments to generate revenue for their creators.

DeepNude uses an AI technique known as Generative Adversarial Networks, or GANs, to generate faux nudes, with the resulting images varying in quality. Most are obviously fake, with smudged or pixelated flesh, but some can easily be mistaken for real pictures.

Since before the advent of Photoshop, people have been creating non-consensual fake female nudes. There are currently many forums and websites dedicated to this activity using non-AI tools, with users sharing nudes of celebrities and people they know. But deepfakes led to faster generation of more realistic images. Now, the automation of this process via robots Telegram makes generating fake nudes as easy as sending and receiving images.

“The main difference is the accessibility of this technology,” said Giorgio Patrini, CEO of Sensity and co-author of the report. The edge. “It's important to note that other versions of this bot's AI core, image processing and synthesis, are available for free from online code repositories. But you have to be a programmer and have some understanding of computer vision to make them work, other than powerful hardware. At the moment, all of this doesn't matter as it is taken care of by the bot built into a messaging app.

The Sensity report states that it is "reasonable to assume" that most people using these bots "are primarily interested in consuming deepfake pornography" (which remains a popular category on pornographic sites). But these images and videos can also be used for extortion, blackmail, harassment, etc. There have been a number of documented case women targeted using AI-generated nudes, and it's possible that some of those who create nudes using the robots Telegram do so with these reasons in mind.

Patrini said The edge that Sensity researchers had not seen direct evidence of the bot's creations being used for these purposes, but said the company believed it was happening. He added that if the political threat of deepfakes had been "miscalculated" ("from the authors' point of view, it is easier and cheaper to resort to photoshop of images and achieve a similar impact to disseminate disinformation, with less effort ”), it is clear that technology poses“ a serial threat to personal reputation and security ”.

telegram telegram Fri