What Inkling 2 Small’s Benchmark Gains Mean for AI Work

Thinking Machines’ Inkling 2 Small posts much higher scores than its predecessor, especially on agentic tasks, visual benchmarks and coding evaluations. Its stronger results come with more output tokens and a higher per-task cost, while the model’s weights are not yet available for download.

WTF Index TERMINATOR
◄ Terminator 1 Idiocracy 0 ►

The story reports stronger agentic capabilities for multi-step work, a mild Terminator lean, while noting higher cost and limited availability.

What Inkling 2 Small’s Benchmark Gains Mean for AI Work

Thinking Machines has introduced Inkling 2 Small, a multimodal model designed as a base for fine-tuning. It makes a large jump over the first Inkling Small on several reported benchmarks, with particularly notable gains on tasks that involve carrying out multi-step work. Those improvements come with higher token use and cost, and the model is currently offered through the company’s Tinker platform.

A larger model with a selective workload

Inkling 2 Small is a mixture-of-experts model with 594 billion parameters in total, of which 17 billion are active at any given time. It accepts text and images, and users can adjust its thinking depth to favor speed or more thorough responses.

The company describes Inkling models as starting points for fine-tuning. That design may suit teams building specialized applications, though benchmark results alone do not establish how well a model will perform in any particular deployment.

On the Artificial Analysis Intelligence Index, Inkling 2 Small scores 31, compared with 13 for Inkling Small. The article says that puts it roughly on par with Google’s Gemini 3.8 Flash.

Agentic results rise, with a reliability trade-off

The clearest progress is on agentic evaluations, which measure a model’s ability to execute multi-step workflows. Inkling 2 Small moves from 5 to 46 percent on AutomationBench-AA and from 5 to 25 percent on FrontierCode 2.0. On AA-Briefcase, a knowledge-work benchmark, it records 1,527 Elo, 615 points above its predecessor.

Another reported change needs careful interpretation: the hallucination rate falls from 61 to 27 percent, but the model also answers fewer questions. Its response rate drops from 77 to 53 percent. A lower hallucination rate alongside a lower response rate does not by itself show that the model has become more reliable on questions it does answer.

Thinking Machines says it used reinforcement learning on hundreds of thousands of long-horizon coding tasks. An automated evaluation system checked whether the code worked and how the model approached each task. The article reports scores of 25.4 percent on FrontierCode 2.0, 72.1 percent on DeepSWE and 86.3 percent on SWEBench Pro v2 (Hard). It also notes that Deepseek-V4. 1-Flash scores higher on FrontierCode, at 28.5 percent, and on DeepSWE, at 77 percent.

Visual and design benchmarks add another strength

The model also accepts images and shows strong results on several visual evaluations. On GDP.pdf, a test of question answering from complex PDF documents, it scores 25.2 percent, the highest among the models compared in the article. It reaches 81.6 percent on MMMU Pro, which evaluates multimodal understanding of academic tasks involving diagrams and figures.

On PerceptionBench, a visual perception evaluation, Inkling 2 Small scores 64.7 percent. For website creation, it earns 1,340 Elo on DesignArena Website, where users compare AI-generated websites head to head. The article identifies this as the highest score among open-weight models. Thinking Machines also says the model has improved at 3D modeling.

The company offered a more unusual demonstration: in a blind wine tasting based on photos, the model correctly identified three wines in 19 out of 20 attempts. That result is a company demonstration, distinct from the benchmark scores and the claims they support.

More capable, but more expensive and not yet downloadable

Inkling 2 Small produces roughly 150,000 output tokens per task on average, about four times the amount produced by its predecessor. At $1.18 per task, it costs nearly six times as much as the earlier model, according to Artificial Analysis. The article compares that price with $7.77 for Gemini 3.8 Flash and $10.02 for Claude Sonnet 5.5.

Thinking Machines says its full training run processed about 12 trillion tokens. The reinforcement-learning stage used 3 trillion tokens on 2,048 GPUs over nine days.

The model has limitations as well as strengths. The company says it can communicate inefficiently, may not choose the best tools for large codebases, does not support audio and is less optimized for chat. It can also overreact to health-related topics and produce cluttered output.

For now, access is through Tinker. Thinking Machines plans to release the weights under an Apache 2.0 license after external safety testing is complete, and says more sizes in the Inkling family are planned. Until then, teams considering the model can weigh its benchmark results against its cost, response behavior and current access limits.