The social media network X has caused widespread outrage after people utilized its AI chatbot Grok to alter photos of females by stripping their attire.
Numerous instances have surfaced where the technology has been employed to digitally undress women to show them in swimwear non-consensually, as well as to place them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to questions, except for an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to prohibit nudification technology. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and significant fines".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the potential" of people seeing unlawful material on their platforms, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' prompts when they reference it in a post.
It is frequently used to provide reactions or further background to other posters' comments, but people on X are also capable of edit an uploaded image through its AI image editing tool.
Background of Criticism and Policy
The chatbot has been criticised for allowing users to create pictures and footage with explicit content and sexually suggestive content, and it was previously accused of generating inappropriate content of celebrities.
"The platform has been permitting the generation and spread of these images for a long time without taking any action and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing likenesses of individuals in a sexually explicit way".
In a comment, Ofcom said it was against the law to "make or circulate private pictures or illegal content" and stated this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the danger" of users coming across unlawful material on their services and to delete it quickly when they become aware of it.
