A German recipe asks for “ein Pfund Mehl”. Most people, and most AI models, read that as one pound of flour. In Germany, a Pfund is 500 grams, about 10% more than an imperial pound. Gemini 3.1 Pro translated it as “a pound of flour”. The sentence reads perfectly. The cake comes out wrong.
Errors like this slip through because the output looks fluent. The Last Translation Benchmark was built to collect exactly these cases. As of 20 October 2026, Lara 3 Think, the reasoning model in Lara Translate, tops its leaderboard with a 56.09% pass rate. Here’s how the benchmark works, how the scores compare, and when reasoning is worth the extra seconds.
TL;DR
- What: The Last Translation Benchmark is an academic test of AI translation, and its 911-item evaluation set is built from sentences that AI translators get wrong.
- Result: Lara 3 Think tops the Blind leaderboard at 56.09%, almost 11 points ahead of GPT-6 Astra Pro (45.33%).
- Also: Lara 2 Think Pro ranks third at 44.90%, so Lara Translate holds two of the top three spots.
- Bottom line: On slang, idioms and cultural references, a reasoning step before translating is what separates the top system from the rest.
SHORT ANSWER
Lara 3 Think scores 56.09% on the Last Translation Benchmark (LTBv1-eval, Blind mode), the highest pass rate of any AI system on the leaderboard as of 20 October 2026. GPT-6 Astra Pro is second at 45.33%, and Lara 2 Think Pro is third at 44.90%. Every item in the benchmark was chosen because leading AI translators failed it, so the gap shows how well each system handles slang, idioms and cultural references.
Why it matters: The hardest translation errors are the fluent ones: a “pound” of flour that should be 500 grams, an idiom rendered word for word. Nobody flags them until a customer or a court does. A benchmark built solely from these traps reveals which systems actually understand meaning, and the results show that a reasoning step before translation is what closes the gap.
What is the Last Translation Benchmark?
The Last Translation Benchmark (LTB) is an academic project led by Vilém Zouhar, with ETH Zurich among its core institutions. It brings together 260 contributors from 177 institutions. Its first release, LTBv1, contains 3,456 human-written, peer-reviewed examples across 109 languages, covering text, images, audio, and video.
The selection rule is strict. An example was accepted only if most leading machine translation systems got it wrong. Why so harsh? The authors argue that existing translation benchmarks are close to saturation, so a harder test was needed to separate the best systems.
The result leans heavily on informal and culture-bound language. Across the full dataset, the paper’s labels include 986 cultural artifacts and 453 slang items. The leaderboard scores models on LTBv1-eval, a 911-item, text-only subset.
How the Last Translation Benchmark is scored
Every example comes with pass/fail rules written by the contributor who submitted it. A rule might say that “Pfund” has to become 500 g, or that a proverb has to keep its figurative meaning. A translation passes only if it meets every rule.
An LLM checks each translation against those rules. The live leaderboard currently uses Gemini 3.8 Flash as the verifier, while the paper’s interactive verification used Gemini 3.1 Pro. The paper reports a Kendall τb of 90.5 (on a 0 to 100 scale) between the verifier’s system rankings and human rankings. It also recommends open-source LLMs as verifiers for the highest reproducibility.
Funding is disclosed, too. The paper thanks Cohere, Google, and OpenAI for financial support to run the platform at scale, and Charles University for server infrastructure.
The leaderboard has two modes. Every figure in this article comes from the main Blind mode, where models translate without seeing the rules or the human reference.
Last Translation Benchmark results (October 2026)
Top AI systems on LTBv1-eval, Blind mode, as of 20 October 2026. Source: LTB leaderboard.
| Rank | Model | Company | Pass rate |
|---|---|---|---|
| 1 | Lara 3 Think | Translated | 56.09% |
| 2 | GPT-6 Astra Pro | OpenAI | 45.33% |
| 3 | Lara 2 Think Pro | Translated | 44.90% |
| 4 | GPT-6 Astra | OpenAI | 44.02% |
| 5 | Gemini 4 Argon | 40.94% | |
| 6 | Gemini 3.8 Flash | 39.85% | |
| 7 | Gemini 3.1 Pro | 39.30% | |
| 8 | Claude Fable 5.1 | Anthropic | 32.93% |
Human translations by the benchmark contributors score 99.89%, which sets the ceiling.
Lara 3 Think leads the next system by almost 11 percentage points. That’s roughly 98 more sentences translated correctly out of 911, on a test where every item was picked because leading systems failed it. Lara 3 Think ran with the Fluid style and no adaptation at all: no context, translation memories, glossaries or custom instructions.
A quick note on names. Lara Think is the reasoning mode in Lara Translate. Lara 3 Think is its current version, built on Lara 3, and Lara 2 Think Pro belongs to the previous generation.
Test Lara Think on the sentences that trip up other tools
Paste slang-heavy reviews, idiom-rich marketing copy or a tricky contract clause, and compare the output line by line.
Why do Lara Base and Lara Think score so differently?
The same leaderboard lists an earlier standard Lara model (a November 2025 entry) at 0.66%. That figure says more about how the run went than about the quality of the translations. Our logs from those benchmark runs show a long list of technical errors, caused mainly by differences between the standard input formats Lara Translate expects and the formats used to run the benchmark. The public dataset reflects this: 200 of the 911 evaluation items have no Lara output at all, and every one of them counts as a fail.
We’re working with the benchmark team to fix the integration and run the evaluation again, so the leaderboard reflects what Lara Translate actually delivers.
What is Lara Think and how does it work?
Lara Think is the reasoning model in Lara Translate. Before it commits to a translation, it runs a multi-step linguistic analysis of the source, takes glossaries, style guides, and context into account when they’re available, and then revises its own output.
Lara 3 also introduced a training phase. The model generates alternative translations, an automatic judge scores them, and the model refines itself over millions of cycles. According to Translated, Lara Think “cuts errors nearly in half when higher quality is required.”
The trade-off is time. Lara Think can take up to 10x longer than Lara Base. For developers, a callback can return a fast preliminary translation first while the reasoned version completes.
What kinds of errors does the benchmark catch?
These items come from the LTBv1 evaluation set. Each one shows what the source means and what a frontier model produced.
Examples from LTBv1-eval with Gemini 3.1 Pro output.
| Phenomenon | Pair | Source | What it means | Model output |
|---|---|---|---|---|
| Cultural convention | DE→EN | “ein Pfund Mehl” | 500 g of flour | “a pound of flour” |
| Cultural reference | EL→EN | “υποβρύχιο βανίλια” | a Greek vanilla spoon sweet | “a vanilla submarine” |
| Political idiom | FR→EN | “la petite phrase de Sarkozy” | a barbed remark aimed at a rival | “Sarkozy’s little phrase” |
| Grammar gap | ES→EN | “ni tampoco aquel” | “nor that one over there” | “or that one either” |
None of these outputs is ungrammatical. Each one passes a quick read, and each one changes the meaning. In professional settings, that’s the most expensive kind of error: nobody flags it until a customer, a reader, or a court does.
Who is Lara Think for?
Pick Lara Think when a wrong nuance costs more than a few extra seconds:
- Marketing and brand copy, where tone and wordplay carry the message
- If you handle legal, financial or regulated content, one term can change an obligation
- Publishing and editorial work with dialogue and informal registers
- Reviews, social posts and other user-generated content full of slang
- Building a quality-critical localization pipeline? This is the model for it.
For real-time chat, very high volumes, or latency-sensitive workflows, Lara Base is the better fit. Speed matters more there than the last few hard cases.
How to try Lara Think
Developers enable Lara Think through the Lara Translate API by passing reasoning: true in the options object. The developer documentation covers SDKs, callbacks and pricing.
Run your hardest content through Lara Think
One API option turns reasoning on. See how Lara handles the errors that fluent-looking output usually hides.
The bottom line
Benchmarks built from easy sentences can no longer tell strong translation systems apart. The Last Translation Benchmark can, because every item is a trap. On that test, Lara 3 Think leads by almost 11 points, and the reasoning step is the reason. Use it where a missed nuance is expensive, and keep Lara Base for speed.
Have a valuable tool, resource, or insight that could enhance one of our articles?
Send us an email at press@laratranslate.com
We’ll be happy to review it and consider it for inclusion to enrich our content for our readers! ✍️
FAQ
What is the Last Translation Benchmark?
The Last Translation Benchmark is an academic test of AI translation built from human-written examples that most leading machine translation systems get wrong. Its first release, LTBv1, has 3,456 examples across 109 languages, submitted by 260 contributors from 177 institutions.
Which AI translator scored highest on the Last Translation Benchmark?
Lara 3 Think, the reasoning model in Lara Translate, scored highest with a 56.09% pass rate on the 911-item LTBv1-eval set in Blind mode, as of 20 October 2026. GPT-6 Astra Pro is second at 45.33%.
How is the Last Translation Benchmark scored?
An LLM verifier checks each translation against pass/fail rules written by the contributor, and a translation passes only if it meets all of them. The live leaderboard uses Gemini 3.8 Flash as the verifier.
What is Lara Think?
Lara Think is the reasoning model in Lara Translate. It analyzes the source text in several steps before translating, which improves accuracy on ambiguous, informal and culture-bound content.
What’s the difference between Lara Think and Lara Base?
Lara Base is the default model, optimized for speed and consistency on professional content. Lara Think adds a reasoning layer for higher quality and can take up to 10x longer.
How much does Lara Think cost?
Lara Think costs €2,000 per million source characters on the Pro plan and €1,500 on the Team plan, through the Lara Translate API. It isn’t available on the Free plan.
This article is about
- Lara 3 Think ranks first on the Last Translation Benchmark with 56.09%, almost 11 points ahead of GPT-6 Astra Pro.
- How the benchmark is built: 3,456 examples that leading AI translators failed, from 260 contributors in 109 languages.
- Why an LLM verifier with contributor-written rules scores the results, and who funds the platform.
- Why reasoning, not raw translation training, decides performance on slang, idioms and cultural references.
- When to choose Lara Think over Lara Base, and what it costs through the API.
Sources
- Last Translation Benchmark: leaderboard
- Last Translation Benchmark paper (arXiv 2609.04173)
- LTBv1 dataset on Hugging Face
- Introducing Lara 3 (Translated)
- Translate Text: Lara Translate for Developers
