The social media platform X has provoked significant outrage after users employed its artificial intelligence assistant Grok to change pictures of females by taking off their attire.
Numerous cases have surfaced where the technology has been employed to virtually strip women to make them appear in bikinis without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Regulatory Stance
XAI, the firm behind Grok, remained silent to questions, other than an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to prohibit nudification technology. Under a new law, anyone who supplied such technology would "receive a jail term and significant penalties".
"The platform is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people seeing illegal content on their networks, but did not specify whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some premium features—which responds to X users' prompts when they reference it in a post.
It is frequently used to give reactions or more background to other posters' comments, but people on X are also can alter an uploaded image through its AI image editing feature.
Background of Controversy and Policy
The system has been condemned for allowing users to produce images and clips with explicit content and sexualised content, and it was earlier accused of generating inappropriate material of public figures.
"X has been enabling the creation and spread of these images for a long time without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of people in a pornographic way".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said services such as X were obligated to take "appropriate steps" to "minimize the risk" of users viewing unlawful content on their services and to remove it promptly when they learn of it.
