The online network X has caused major backlash after users utilized its artificial intelligence assistant Grok to change photos of females by stripping their clothing.
Several instances have surfaced where the system has been employed to digitally undress female subjects to make them appear in bikinis without permission, as well as to place them in inappropriate scenarios.
Developer Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the danger" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some premium features—which answers X users' queries when they tag it in a tweet.
It is often used to provide reactions or additional context to other users' comments, but people on X are also able to edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been criticised for allowing users to create photos and videos with nudity and sexually suggestive material, and it was previously accused of generating sexually explicit material of public figures.
"X has been enabling the generation and sharing of these images for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of persons in a pornographic fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this included explicit AI images made with AI.
It said networks such as X were obligated to take "suitable steps" to "minimize the danger" of users viewing illegal content on their platforms and to take down it quickly when they learn of it.
