The market for faster AI inference is heating up. While Cerebras drew investor attention with purpose-built chips in its IPO debut in May, French startup Kog is taking a different path: extracting more performance from the standard datacenter GPUs many enterprises already use.
Kog's pitch is simple in outline but demanding in execution. The company says software optimization can make conventional GPUs much more capable for inference, including chips such as the AMD MI300X and NVIDIA H200 GPUs used in its early demo.
Why Kog Is Focused On Existing GPUs
AI inference has become a practical bottleneck because users and companies increasingly care not only whether a model can produce an answer, but how quickly and cheaply it can do so. Kog is positioning itself around that pressure point.
The company gained attention in May when a tech preview reached the front page of Hacker News. The preview was meant to show that extremely fast single-request decoding could be achieved on standard datacenter GPUs already owned by enterprises.
The work did not extend to laptop GPUs, which disappointed some observers. But the datacenter focus still matters for companies that already have expensive GPU infrastructure and want more performance without moving immediately to specialized inference chips.
CEO Gael Delalleau told TechCrunch that the response went beyond curiosity. Kog received 200 tangible business leads after the launch, a sign that inference speed is not just a technical benchmark but a business concern.
The First Demand Is Coming From Software Workflows
Based on early feedback, Delalleau expects software engineering to be Kog's first use case. The reason is straightforward: professional users of AI coding tools often care deeply about waiting time.
The source article points to veteran Claude Code users, who know that results can sometimes take hours. Anthropic has also shown that speed carries commercial value by charging a price multiple for Claude's Fast Mode.
Kog is aiming at customers who are frustrated by those delays because AI workflows are part of their professional work. For these users, faster AI inference is not a luxury feature. It can affect how much work gets done and how quickly a task moves from prompt to output.
The startup is also working with design partners whose products let users generate games and apps with a prompt. Delalleau said that for those partners, a faster result through the Kog Inference Engine, or KIE, could translate into more revenue.
The Technical Challenge Is Scaling Beyond The Demo
Kog's early performance claim is ambitious. The company has promised “30x faster LLM inference,” and its demo showed 3,000 per-request tokens per second.
But that result came with Laneformer 2B, a purpose-built small model with only some 2 billion parameters that has now been open-sourced. The next hurdle is showing that the same approach can deliver meaningful gains on larger models that customers actually want to use.
Kog learned from prospective customers that the market is not yet ready to fine-tune small models. That feedback changed the company's immediate focus. Since the launch, Kog has concentrated on accelerating development for larger models to match the demand it observed.
That shift raises the stakes. The company now has to prove that optimization techniques demonstrated on a small, purpose-built model can carry over to LLMs, where model size itself can create problems for inference hardware.
Delalleau rejects the idea that GPUs are poorly suited for decoding. His view is that newer GPUs have increasing memory bandwidth, and that the remaining opportunity is to unlock more of it through deeper software work.
A Low-Level Approach To GPU Acceleration
Kog is not the only company betting that software can push GPUs further. ZML, also from France, has released hardware-agnostic software that bypasses Nvidia's CUDA to support fast inference across competing chips.
Delalleau, however, compares Kog more closely to Stanford University lab Hazy Research because of its deeper-level focus on GPU acceleration. The distinction is important: Kog is not merely trying to make inference portable across hardware. It is trying to understand specific GPUs closely enough to extract more speed from them.
That approach reflects Delalleau's background. He studied solid-state physics at France's Ecole Polytechnique and later worked in offensive cybersecurity, also known as white hat hacking. He says the scientific side encourages attention to the laws of physics and the laws of the GPU, while the hacking side pushes the team to reverse-engineer systems at a low level.
There is a cost to that depth. Delalleau said Kog may spend several weeks or even months on each new GPU to study the hardware in detail and conduct GPU engineering research. With a team of 11 people, that naturally limits how many chips the company can support in the near term.
What Kog Still Needs To Prove
Kog's future depends on demonstrating that its method works beyond a controlled early preview. The most important proof point will be a major model running at much higher speed.
Delalleau said that once Kog has implemented its first major model at 10x speed, which he thinks will happen in September, the company will be able to demonstrate customer traction and pursue a Series A.
The company also has longer-term ambitions to turn its methodology into agent-based pipelines that can help support more chips and models. If that works, it could reduce the manual burden of going deep on each new GPU.
Kog may also benefit from Europe's interest in building its own capacity around models and chips. The startup is supported by Scaleway and backed by France's Bpifrance and French Tech 2030's program.
For now, the core question remains narrow and concrete: can Kog move from a striking small-model demo to faster inference on the larger LLMs customers want? If it can, the company could make existing datacenter GPUs more valuable at a moment when AI speed and cost are under intense pressure.