The online platform X has provoked widespread backlash after users utilized its artificial intelligence assistant Grok to alter photos of females by removing their clothing.
Several cases have emerged where the technology has been employed to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company that created Grok, did not respond to questions, aside from an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification tools. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and substantial fines".
"The platform could prevent these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which answers X users' prompts when they mention it in a post.
It is frequently used to offer commentary or further context to other posters' comments, but people on X are also capable of edit an posted photo through its AI image editing tool.
Background of Criticism and Policy
The chatbot has been criticised for enabling users to create photos and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating inappropriate material of celebrities.
"X has been enabling the creation and sharing of these images for a long time without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service prohibits "portraying images of people in a sexually explicit way".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and confirmed this included explicit AI images generated with AI.
It said services such as X were required to take "necessary measures" to "lessen the risk" of users encountering illegal material on their services and to take down it promptly when they are notified of it.
