The online network X has sparked significant outrage after individuals used its artificial intelligence assistant Grok to alter photos of women by stripping their clothing.
Numerous examples have emerged where the technology has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, did not respond to requests for comment, aside from an auto-reply response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to prohibit nudification software. Under a upcoming law, anyone who supplied such tools would "receive a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' queries when they mention it in a post.
It is often used to give commentary or further background to other users' comments, but people on X are also capable of edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The chatbot has been criticised for permitting users to create photos and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in making inappropriate material of celebrities.
"The platform has been enabling the generation and distribution of these pictures for a long time without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary steps" to "reduce the risk" of users coming across unlawful material on their services and to remove it swiftly when they learn of it.
