Microsoft is taking legal action against a group it says built a paid system for abusing its generative AI services. The company alleges the operation was designed to get around safety protections and enable harmful and illicit content creation through Microsoft’s AI platform.
The lawsuit names three individuals accused of running the service, along with seven individuals Microsoft says were customers. All 10 defendants are identified as John Doe because Microsoft says it does not know who they are.
What Microsoft Says The Service Did
According to Microsoft, the three operators created tools specifically meant to bypass safety guardrails in generative AI services. Steven Masada, the assistant general counsel for Microsoft’s Digital Crimes Unit, said the defendants also compromised legitimate accounts belonging to paying customers.
Microsoft’s allegation is that those two pieces were combined into a fee-based platform. The service allegedly allowed people to use Microsoft’s AI systems in ways the company forbids, including generating harmful and illicit content.
Lawyers for Microsoft described the case as an effort to stop a coordinated operation. In the complaint filed in federal court in the Eastern District of Virginia and unsealed Friday, they wrote that Microsoft seeks to disrupt a “sophisticated scheme” built around bypassing AI safety guardrails.
How The Alleged Workaround Operated
The service allegedly sold access through a now-shuttered site at “rentry[.]org/de3u.” Microsoft says it ran from last July to September, when the company took action to shut it down.
The platform included instructions for customers. Microsoft says those instructions explained how to use custom tools to generate harmful and illicit content.
The lawsuit also describes a technical layer built to sit between the service’s customers and Microsoft’s AI systems. Microsoft says the service used a proxy server that relayed traffic between users and the servers providing Microsoft’s AI services.
Among the details in the complaint, Microsoft alleges that the proxy service used undocumented Microsoft network application programming interfaces, or APIs, to communicate with Azure computers. The requests were designed to look like legitimate Azure OpenAPI Service API requests, and they used compromised API keys for authentication.
Microsoft did not say exactly how the legitimate customer accounts were compromised. The company did say hackers have been known to build tools that search code repositories for API keys that developers accidentally include in apps they create. Microsoft also raised the possibility that credentials were stolen by people who gained unauthorized access to the networks where those credentials were stored.
Why Guardrails Are Central To The Case
Microsoft and other generative AI providers ban certain uses of their systems. The source article identifies several types of content that are off limits, including material that features or promotes sexual exploitation or abuse, erotic or pornographic content, and content that attacks, denigrates, or excludes people based on protected or similar traits.
The prohibited categories also include threats, intimidation, promotion of physical harm, and other abusive behavior. These limits matter because Microsoft’s lawsuit is not only about access to accounts. It is also about the alleged effort to defeat the systems put in place to enforce those rules.
Microsoft says its guardrails inspect both user prompts and generated output for signs that a request violates its terms. The company did not outline precisely how the defendants’ software allegedly bypassed those protections.
Masada wrote that Microsoft’s AI services use “strong safety measures” at the model, platform, and application levels. He also said Microsoft observed a foreign-based threat-actor group using sophisticated software that exploited exposed customer credentials scraped from public websites.
According to Masada, the group sought to access accounts connected to certain generative AI services and alter the capabilities of those services. He said cybercriminals then used the services and resold access to other malicious actors, along with detailed instructions for generating harmful and illicit content.
What Microsoft Wants From The Court
The complaint makes a broad set of legal claims. Microsoft alleges violations of the Computer Fraud and Abuse Act, the Digital Millennium Copyright Act, the Lanham Act, and the Racketeer Influenced and Corrupt Organizations Act.
The lawsuit also alleges wire fraud, access device fraud, common law trespass, and tortious interference. Microsoft is seeking an injunction that would stop the defendants from engaging in “any activity herein.”
The case shows how generative AI abuse can involve more than a single prompt or a single user. As Microsoft describes it, the alleged operation combined compromised credentials, proxy infrastructure, undocumented APIs, custom tools, and paid access into a service aimed at bypassing platform controls.
For developers and companies using AI services, the account-access part of the complaint is especially important. Microsoft and others have long advised developers to remove credentials and sensitive data from published code, but the source article notes that this practice is regularly ignored.
Microsoft says that after discovering the activity, it revoked cybercriminal access, put countermeasures in place, and enhanced safeguards to block similar malicious activity in the future.