ZML is trying to make AI infrastructure less dependent on any single chip stack. The French startup has released ZML/LLMD, a free LLM inference server built to run a variety of open-source large language models on a variety of chips.
The release arrives at a time when inference, the processing of prompts, is becoming a central problem for AI deployment. ZML founder Steeve Morin told TechCrunch that software and architecture barriers can make the process feel fragmented and can create vendor lock-in behind the scenes.
Why inference is becoming the pressure point
Training models remains important, but the source article says optimizing inference has been outpacing model training in importance as AI becomes part of work and everyday life. That shift changes what matters for companies using AI systems at scale.
Inference is where prompts are processed. If that layer is slow, expensive, or restricted to a narrow set of hardware choices, AI adoption can become harder to justify. ZML/LLMD is aimed at this practical layer: how to get large language models running efficiently once they are already available for use.
Morin described the current environment as one where software and architecture barriers can make AI infrastructure patchy. The implication is clear: even when capable chips exist, they may not be easy to use together, and the available performance may be limited by the surrounding software.
What ZML/LLMD is designed to do
ZML/LLMD is an LLM inference server. According to the source, it allows a variety of open-source large language models to run on a variety of chips, including Nvidia’s, AMD’s, Google’s TPU, Apple Metal and Intel Arc.
The company’s broader goal is to break existing silos. Morin told TechCrunch that ZML wants to make different chips available for AI use cases at their maximum available speed, and sometimes faster.
That is both a technical claim and a market claim. If software can help teams use different AI chips more effectively, enterprises and clouds could have more options when building AI systems. The source says ZML hopes to provide the option to use a mix of chips, including some that might be less costly or consume less energy.
“The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated,” Morin said.
The key point is flexibility. ZML is not only presenting ZML/LLMD as a speed tool, but as a way to loosen the connection between AI software and a single hardware path.
The chip market context
The source makes clear that Nvidia’s market position remains strong. It says the days of Nvidia’s unparalleled market dominance are not over, but that challengers and choices are appearing from many directions.
Morin is not bearish on Nvidia. The article says that is partly because of Nvidia’s existing supply, and that ZML has a good relationship with the AI chip giant. Nvidia has also been gearing up for the rise of inference.
At the same time, ZML’s software could help newer AI chipmakers. Morin cited Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA, and observed that many such companies happen to be from Europe.
For Morin, the region is not the main point. What matters, he told TechCrunch, is that ZML can work with these companies on “things that haven’t been done before anywhere in the world.”
Competition in the inference gold rush
Inference is drawing intense investment. The source says the trend has been called the “inference gold rush,” and ZML is entering a field with several visible competitors.
Those competitors include Baseten, recently valued at $13 billion; Inferact, from the creators of open source project vLLM; and RadixArk, the commercial company behind SGLang. Both vLLM and SGLang partially compete with LLMD.
ZML’s ambitions go beyond that immediate overlap. Morin told TechCrunch, “We have reached the point where we are co-designing silicon.” That suggests the company sees its work not only as server software, but as part of a wider hardware and software design process.
The startup is also small. Morin credited ZML’s lean team of 20 people as a reason the Paris-based company has been able to move quickly, with more releases planned.
Free, but not open source
ZML/LLMD is launching as a free product, but it is not open source. That differs from ZML’s first public project, an inference-focused ML framework released in 2024 and updated in March.
Morin said the free launch is meant to help the company learn from usage before deciding where revenue should come from.
“I’d rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go,” Morin said.
The timing and shape of a paid version remain unclear. The source says it is too early to know when ZML/LLMD might become a paid product or what adoption will look like.
Still, ZML has attracted attention from notable investors and founders. Morin raised $20 million from venture firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.
The company’s cap table also includes Dagger and Docker founder Solomon Hykes, Clément Delangue and Julien Chaumond from Hugging Face, and LeCun, now with AMI Labs. ZML was endorsed by Turing Award winner Yann LeCun.
For Morin, ZML is also tied to place. The source frames the company as part of a case that Europe’s AI startups can now build from home. “I couldn’t do ZML anywhere but in Paris,” Morin said.