Elon Musk's online platform X has sparked widespread anger after people utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their garments.
Several examples have been seen where the tool has been used to remove clothing from women to show them in swimwear without permission, as well as to place them in inappropriate contexts.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, offered no comment to questions, other than an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being brought forward to ban image stripping software. Under a upcoming criminal offence, anyone who distributed such technology would "receive a prison sentence and heavy penalties".
"The platform has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people seeing illegal content on their services, but did not specify whether it was at this time examining X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they reference it in a message.
It is frequently used to give commentary or additional context to other posters' remarks, but people on X are also able to modify an uploaded image through its AI image editing tool.
History of Controversy and Rules
The system has been criticised for allowing users to produce pictures and footage with explicit content and sexually suggestive material, and it was previously alleged to have been involved in making sexually explicit content of celebrities.
"X has been permitting the production and sharing of these images for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules forbids "depicting images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said services such as X were required to take "necessary measures" to "reduce the danger" of users encountering unlawful material on their platforms and to remove it promptly when they learn of it.
