Elon Musk's social media platform X has caused major outrage after users employed its artificial intelligence assistant Grok to manipulate photos of women by taking off their attire.
Numerous instances have surfaced where the system has been employed to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexually explicit scenarios.
Developer Response and Legal Stance
XAI, the company that created Grok, offered no comment to questions, aside from an standard response stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification technology. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and significant fines".
"The platform is capable of halting these types of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their services, but stopped short of confirming whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they mention it in a message.
It is commonly used to give reactions or additional context to other users' remarks, but people on X are also can modify an uploaded image through its artificial intelligence picture modification feature.
History of Criticism and Policy
The tool has been criticised for permitting users to create images and footage with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in creating inappropriate material of celebrities.
"X has been enabling the generation and sharing of these pictures for a long time without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing images of people in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or illegal content" and confirmed this applied to explicit AI images made with AI.
It said platforms such as X were required to take "suitable measures" to "reduce the risk" of users coming across unlawful material on their platforms and to remove it quickly when they are notified of it.
