AI applications for the abusive creation of sexualized deepfakes are to be banned in the EU in the future.
May 7, 2026, 7:43 amMay 7, 2026, 7:43 am
Representatives of the member states and the European Parliament agreed on a corresponding adjustment of the AI law, as the Cypriot EU Council Presidency announced. At the same time, other AI rules are to be simplified in order to exploit the economic potential of artificial intelligence (AI) in Europe.
Elon Musk’s AI Grok was able to temporarily create nude deepfakes. Image: EPA
Before the changes can come into force, the agreement must be confirmed by the plenary session of the EU Parliament and the Council of Member States’ Governments. As a rule, this is considered a formality. If the reform goes through, the ban would be enforced from December 2, 2026 by the EU’s AI Office, which was created two years ago.
In so-called deepfakes, a woman’s face is inserted into another video – such as a porn sequence – or her voice is artificially imitated so that it appears as if she is doing or saying things that never actually happened. The development of AI has made creating deceptively real content much easier.
The new ban should also explicitly target the creation of content that depicts child sexual abuse. The FDP MEP Svenja Hahn welcomed the agreement: “AI must not be a tool for sexual violence against children,” she said after the negotiations that lasted late into the night.
Grok scandals and debate about digital violence
At the EU level, the issue came into focus at the end of last year thanks to the AI chatbot Grok: Until the US company behind the software, run by tech billionaire Elon Musk, restricted this function, people repeatedly ordered the AI to undress women in images they selected. On New Year’s Eve, the chatbot itself apologized for creating an image of two teenage girls “in sexualized outfits.”
In Germany, the debate about sexualized digital violence took on new urgency at the end of March when Collien Fernandes went public with allegations against her ex-husband Christian Ulmen. She accuses him of creating fake profiles in her name and using them to distribute pornographic content. The presumption of innocence applies to elms.
Switzerland is also discussing deepfakes
The discussion about digital and sexual violence against women is also taking place in Switzerland. According to the Swiss association “Tech Against Violence,” the “non-consensual creation and distribution of intimate and sexualized images and videos” has increased significantly thanks to AI. This has serious consequences for those affected – regardless of whether the recordings are real or manipulated.
AI-generated sexualized content is “part of a continuum of gender-based violence,” explained Simone Eymann from “Tech Against Violence” in an interview with the Keystone-SDA news agency a month ago. In 99 percent of cases, women and girls are affected.
On International Women’s Day, March 8th, National Council President Pierre-André Page (SVP/FR) emphasized Parliament’s responsibility to protect against digital violence. Switzerland follows European legislation and has to define its own rules. Technology must serve people, he said.
With the Federal Act on Communication Platforms and Search Engines, the Federal Council wants to create more transparency and security in the digital space. However, the consultation showed that most of those involved in the consultation did not go far enough. The MAZ media training center in Lucerne, for example, called for strict labeling requirements for content created by AI – so that deepfakes can be recognized.
The EU has already presented a directive on the subject
The planned tightening of the law is not the first EU rules to combat digital violence. Existing regulations already stipulate that the creation and distribution of manipulated depictions of sexual acts without the consent of those affected should be punishable in all member states.
The new ban in the AI regulation at EU level that is now emerging would change the perspective from punishing the crime to banning the tool, i.e. the AI application itself. Participants in the negotiations emphasize that the ban is not intended to excessively restrict the creation or manipulation of images.
The new law should precisely define what is sexualized content and what is not. Non-consensual bikini pictures like those created by Grok and distributed on X could therefore remain permitted.
Mandatory watermarks
The EU Commission originally proposed changes to the AI law in order to relieve the burden on the economy and in particular the AI industry. However, companies have recently repeatedly asked for more time to implement the necessary adjustments to meet the stricter requirements. According to the agreement, providers of chatbots and other services should be given this time.
The companies behind ChatGPT, Claude and Co. now only have to comply with parts of the law that the European AI Office was originally supposed to enforce from August onwards from December 2026. By then at the latest, providers should clearly mark AI content as such – so they would have to mark generated images and videos with watermarks. The AI office is only scheduled to enforce other rules from December 2027. (sda/dpa)