🔗 Share this article

The online network X has sparked significant anger after people employed its artificial intelligence assistant Grok to manipulate images of females by removing their attire.

Multiple cases have emerged where the tool has been used to virtually strip women to show them in bikinis without consent, as well as to put them in inappropriate contexts.

Developer Response and Regulatory Position

XAI, the organization that created Grok, did not respond to questions, aside from an automatically-generated message stating "legacy media lies".

A government representative stated that new laws is being brought forward to outlaw nudification software. Under a proposed criminal offence, anyone who provided such technology would "be subject to a jail term and substantial fines".

"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."

Regulator Position and Platform Capabilities

The regulator Ofcom said tech firms must "assess the danger" of people accessing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in relation to AI-generated images.

Grok is a free AI assistant—with some premium functions—which responds to X users' requests when they tag it in a post.

It is commonly used to give reactions or further context to other posters' comments, but people on X are also able to edit an uploaded image through its AI image editing tool.

History of Controversy and Rules

The tool has been condemned for allowing users to produce pictures and footage with nudity and sexualised content, and it was in the past accused of generating graphic content of public figures.

"The platform has been enabling the generation and sharing of these images for months without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.

XAI's own terms of service bans "showing images of people in a pornographic fashion".

In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.

It said networks such as X were required to take "necessary actions" to "reduce the risk" of users coming across illegal material on their services and to remove it swiftly when they are notified of it.