Elon Musk's online platform X has caused significant anger after individuals utilized its AI chatbot Grok to change photos of women by removing their clothing.
Several examples have been seen where the tool has been employed to digitally undress women to show them in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Response and Regulatory Position
XAI, the firm that created Grok, offered no comment to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to outlaw nudification software. Under a upcoming law, anyone who distributed such tools would "receive a jail term and significant fines".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their platforms, but did not specify whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' prompts when they tag it in a post.
It is frequently used to offer reactions or additional context to other posters' comments, but people on X are also can modify an uploaded image through its AI image editing feature.
History of Criticism and Rules
The tool has been faulted for permitting users to generate photos and footage with explicit content and sexually suggestive material, and it was earlier accused of generating sexually explicit material of celebrities.
"X has been permitting the generation and sharing of these images for months without taking any steps and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said networks such as X were mandated to take "necessary actions" to "minimize the risk" of users viewing unlawful material on their services and to take down it swiftly when they learn of it.
