State officials are moving quickly against xAI after Grok users generated sexualized images, including nonconsensual intimate images and images involving children. The response now spans at least 37 attorneys general for US states and territories, according to WIRED.
The issue is no longer only about one chatbot feature. It has become a broader test of how states expect AI companies, social platforms, payment processors, search engines, and lawmakers to respond when generative tools are used to create harmful sexual imagery.
A coordinated warning to xAI
On Friday, a bipartisan group of 35 attorneys general published an open letter to xAI. The letter demanded that the company “immediately take all available additional steps to protect the public and users of your platforms, especially the women and girls who are the overwhelming target of [non-consensual intimate images].”
WIRED also reported that attorneys general from California and Florida said they had taken action, bringing the total number of state and territory attorneys general involved to at least 37.
The letter arrived after Grok drew international regulatory attention for the creation of intimate deepfake images of people without consent and sexualized images of children. The concern includes both Grok’s account on X and the Grok Imagine model available through the Grok website.
X did not respond to WIRED’s request for comment. xAI responded to WIRED’s questions with, “Legacy Media Lies.”
What officials say Grok allowed
A recent report from the Center for Countering Digital Hate estimated that during an 11-day period starting on December 29, Grok’s account on X generated around 3 million photorealistic sexualized images. The same report estimated that around 23,000 of those were sexualized images of children.
WIRED previously reported that people were also creating far more explicit videos using the Grok Imagine model on the Grok website. Unlike X, the Grok site did not appear to require any sort of age verification before people could view content.
The attorneys general said Grok’s ability to create nonconsensual sexual imagery had been used as a “selling point” by xAI. They also said xAI claims it has stopped Grok’s X account from undressing people, but argued that the company has not removed nonconsensually created content, “despite the fact that you will soon be obligated to do so by federal law.”
The letter called on xAI to take several specific steps:
- Remove Grok’s ability to depict people in revealing clothing or suggestive poses.
- Suspend offending users and report them to authorities.
- Give users the ability to control whether their content can be edited by Grok.
- Remove nonconsensually created content.
The letter also said attorneys general have pursued investigations and prosecutions in this area, called on payment processors and search engines to reduce the creation of nonconsensual intimate images, and advocated for legislation to prevent AI-powered child exploitation.
State investigations and direct action
Some attorneys general contacted by WIRED said they were investigating or discussing concerns with X or xAI specifically around Grok being used to generate child sexual abuse material, or CSAM. According to the child advocacy group Enough Abuse, 45 states prohibit AI-generated or computer-edited CSAM.
Arizona attorney general Kris Mayes opened an investigation into Grok on January 15, according to Richie Taylor, her communications director. In a news release about the investigation, Mayes described reports about the imagery being created as “deeply disturbing.” She also called on Arizonans who believe they were victimized by Grok to contact her office.
“Technology companies do not get a free pass to create powerful artificial intelligence tools and then look the other way when those programs are used to create child sexual abuse material. My office is opening an investigation to determine whether Arizona law has been violated,”
California attorney general Rob Bonta sent a cease and desist letter to Elon Musk on January 16. The letter demanded that xAI take immediate action to stop the creation and distribution of CSAM or nonconsensual intimate images through both the Grok account on X and the stand-alone Grok app.
Elissa Perez, press secretary for the California Department of Justice, told WIRED that xAI had formally responded. She said the office now has “reason to believe, subject to additional verification, that Grok is not currently being used to generate any sexual images of children or images that violate California law.” California’s investigation is still ongoing.
Florida is also engaged. Jae Williams, press secretary for the Florida Attorney General's Office, told WIRED that the office is “currently in discussions with X to ensure that protections for children are in place and prevent its platform from being used to generate CSAM."
Missouri signaled a compliance-focused stance as well. Stephanie Whitaker, director of communications for the Missouri Attorney General’s Office, said the state “has a duty to ensure X and other social media companies comply with state law. Companies profiting off of an oasis for criminal activity may find themselves culpable."
Age verification creates a hard policy question
The Grok controversy is unfolding as half the country has already passed age verification laws. Those laws require people looking at pornography to provide proof that they are not a minor.
WIRED contacted attorney general offices in the 25 states that have passed age verification laws to ask how they were responding to nonconsensual sexualized images on Grok and X. WIRED also contacted sponsors of age verification bills in those states to ask whether X or xAI should do more to stop children from viewing explicit content on X and Grok.com.
The harder question is how those laws might apply to platforms such as X, which were not their intended target. Almost every state with age verification has followed Louisiana’s 2022 approach, which requires more than one-third of a site’s content to be pornographic or harmful to minors before restrictions apply.
That creates a practical problem: regulators must decide what counts as a piece of content and whether that content is pornographic. Alan Butler, executive director of Electronic Privacy Information Center, previously told WIRED, “It's mostly a counting question in terms of ‘does the law apply’”.
Lawmakers are widening the focus
Child safety and AI are already a priority for state lawmakers. In early December, 42 attorneys general cosigned a letter to AI companies, including xAI, asking them to “adopt additional safeguards to protect children.”
On January 14, a working group with representatives from many of the same attorney general offices met to discuss emerging AI issues. During that meeting, North Carolina attorney general Jeff Jackson said AI-generated CSAM “should be an early priority.” Jackson also signed Friday’s letter.
Arizona state representative Nick Kupper, a Republican, told WIRED he recently filed a bill that would require sites posting explicit content, including AI-generated imagery, to verify the ages and consent of performers before doing so. The state criminalized AI-generated CSAM last year.
Georgia Senate majority leader Jason Anavitarte said the state is “actively working to put additional protections in place alongside existing statutes” with respect to consensual sexual imagery generated by AI.
“Legislation will be introduced this session so that obscene material, including AI-generated sexual material involving minors, can be criminally prosecuted in Georgia, with a specific focus on those who create the imagery using AI tools,”
The pressure on xAI shows how fast generative AI safety has moved from platform policy to state enforcement. The central demand from officials is straightforward: if a company releases powerful image tools, states expect it to prevent abuse, remove harmful content, and respond when those tools are used against women, girls, and children.