The online platform X has sparked major anger after individuals utilized its AI chatbot Grok to manipulate pictures of women by taking off their clothing.
Several cases have surfaced where the tool has been employed to virtually strip women to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.
Company Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to inquiries, aside from an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to prohibit image stripping technology. Under a new criminal offence, anyone who distributed such tools would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people seeing illegal content on their networks, but did not confirm whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' prompts when they mention it in a tweet.
It is often used to give commentary or more background to other posters' comments, but people on X are also can modify an posted photo through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been criticised for permitting users to produce photos and clips with nudity and sexually suggestive content, and it was in the past accused of creating sexually explicit content of celebrities.
"X has been allowing the generation and distribution of these images for a long time without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of persons in a graphic way".
In a statement, Ofcom said it was illegal to "make or share private pictures or illegal content" and stated this covered sexual deepfakes generated with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the danger" of users viewing illegal content on their services and to remove it swiftly when they become aware of it.
