🔗 Share this article

The social media network X has provoked significant backlash after users used its AI chatbot Grok to manipulate images of females by removing their attire.

Multiple examples have been seen where the technology has been employed to digitally undress women to show them in bikinis without consent, as well as to place them in inappropriate contexts.

Developer Response and Regulatory Stance

XAI, the company that created Grok, remained silent to inquiries, other than an auto-reply message stating "traditional news outlets lies".

A Home Office official stated that legislation is being brought forward to prohibit nudification technology. Under a proposed criminal offence, anyone who provided such technology would "receive a jail term and substantial penalties".

"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."

Regulator Position and Platform Capabilities

The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing illegal content on their services, but did not confirm whether it was at this time looking into X or Grok in relation to AI-generated images.

Grok is a no-cost chatbot—with some premium functions—which answers X users' queries when they tag it in a post.

It is frequently used to provide commentary or additional background to other users' remarks, but people on X are also can alter an posted image through its artificial intelligence picture modification tool.

Background of Criticism and Rules

The tool has been criticised for allowing users to produce images and clips with explicit content and sexualised content, and it was previously accused of creating graphic material of public figures.

"X has been permitting the production and sharing of these images for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.

XAI's own terms of service forbids "showing images of people in a pornographic fashion".

In a comment, Ofcom said it was illegal to "create or distribute private pictures or illegal content" and confirmed this included explicit AI images made with AI.

It said networks such as X were required to take "appropriate measures" to "reduce the risk" of users coming across illegal content on their services and to delete it swiftly when they are notified of it.