The online platform X has caused widespread outrage after individuals utilized its AI chatbot Grok to alter pictures of females by removing their attire.
Several instances have surfaced where the system has been used to remove clothing from women to make them appear in bikinis non-consensually, as well as to put them in sexually explicit scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, did not respond to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit image stripping technology. Under a proposed law, anyone who distributed such technology would "be subject to a prison sentence and heavy fines".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people viewing illegal content on their networks, but did not confirm whether it was at this time investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some paid-for features—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to offer reactions or further background to other users' remarks, but people on X are also able to modify an posted picture through its AI image editing function.
Background of Criticism and Rules
The tool has been criticised for enabling users to create images and footage with nudity and sexualised material, and it was previously alleged to have been involved in making sexually explicit content of public figures.
"X has been enabling the creation and sharing of these images for months without taking any steps and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own usage rules bans "depicting images of individuals in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the danger" of users viewing unlawful material on their services and to take down it promptly when they become aware of it.
