The online platform X has provoked widespread backlash after individuals employed its AI chatbot Grok to change pictures of females by removing their garments.
Several cases have surfaced where the system has been employed to digitally undress women to make them appear in bikinis without permission, as well as to put them in sexually explicit situations.
Developer Response and Legal Position
XAI, the organization that created Grok, offered no comment to questions, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to prohibit image stripping software. Under a upcoming law, anyone who provided such technology would "receive a jail term and substantial fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people seeing illegal content on their services, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they mention it in a tweet.
It is commonly used to provide reactions or additional background to other users' comments, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been criticised for enabling users to create pictures and clips with nudity and sexually suggestive material, and it was in the past accused of generating sexually explicit material of celebrities.
"The platform has been allowing the creation and sharing of these images for a long time without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of persons in a sexually explicit way".
In a statement, Ofcom said it was against the law to "generate or circulate deepfakes or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said networks such as X were mandated to take "suitable measures" to "minimize the danger" of users encountering unlawful content on their services and to delete it quickly when they are notified of it.
