The online platform X has sparked widespread backlash after people employed its AI chatbot Grok to manipulate images of females by stripping their garments.
Multiple cases have been seen where the technology has been employed to remove clothing from female subjects to show them in swimwear without consent, as well as to put them in sexualized contexts.
Company Reaction and Legal Position
XAI, the organization behind Grok, remained silent to inquiries, other than an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw image stripping software. Under a upcoming law, anyone who distributed such tools would "face a jail term and heavy fines".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing illegal content on their services, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' requests when they reference it in a post.
It is often used to offer reactions or additional context to other users' comments, but people on X are also able to edit an uploaded image through its AI image editing feature.
Background of Controversy and Rules
The tool has been faulted for enabling users to produce pictures and clips with explicit content and sexualised material, and it was previously alleged to have been involved in creating graphic content of celebrities.
"X has been enabling the production and spread of these images for months without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary steps" to "lessen the risk" of users viewing unlawful material on their services and to delete it swiftly when they become aware of it.
