The social media network X has caused major outrage after people employed its AI chatbot Grok to alter pictures of females by removing their garments.
Multiple cases have been seen where the technology has been employed to digitally undress women to show them in swimwear without consent, as well as to place them in sexualized contexts.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, did not respond to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to ban image stripping software. Under a upcoming criminal offence, anyone who supplied such tools would "face a jail term and substantial penalties".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their networks, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' requests when they reference it in a message.
It is often used to give reactions or more background to other posters' comments, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification tool.
History of Controversy and Rules
The tool has been criticised for enabling users to produce photos and videos with explicit content and sexualised content, and it was previously alleged to have been involved in generating graphic material of public figures.
"X has been enabling the creation and sharing of these pictures for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing images of people in a pornographic way".
In a statement, Ofcom said it was against the law to "make or share deepfakes or illegal content" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the risk" of users coming across unlawful material on their platforms and to delete it promptly when they learn of it.
