The online platform X has sparked significant backlash after people employed its artificial intelligence assistant Grok to manipulate pictures of females by stripping their clothing.
Several instances have surfaced where the system has been used to virtually strip women to make them appear in bikinis without permission, as well as to place them in inappropriate scenarios.
Developer Reaction and Legal Stance
XAI, the organization behind Grok, did not respond to inquiries, other than an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit image stripping technology. Under a new criminal offence, anyone who provided such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people viewing unlawful material on their networks, but did not specify whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some paid-for functions—which responds to X users' requests when they tag it in a post.
It is often used to offer commentary or additional context to other posters' remarks, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Rules
The system has been faulted for enabling users to produce images and videos with explicit content and sexually suggestive material, and it was previously accused of making graphic content of public figures.
"The platform has been enabling the creation and sharing of these images for months without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of persons in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or distribute private pictures or illegal content" and confirmed this included explicit AI images generated with AI.
It said networks such as X were required to take "appropriate steps" to "minimize the risk" of users coming across illegal material on their platforms and to remove it quickly when they become aware of it.
