The authors of BabelArena describe a benchmark for language-model agents across 23 languages and four families of tasks. Their revised paper was published on October 23; the work evaluates multilingual agent behavior rather than announcing a new model.
Multilingual quality is not just a matter of whether an agent eventually returns an answer. The steps, costs and failure patterns along the way can matter to people building services for users in different languages.
What happened?
The authors of BabelArena describe a benchmark for language-model agents across 23 languages and four families of tasks. Their revised paper was published on October 23; the work evaluates multilingual agent behavior rather than announcing a new model.
In one paired shopping task, the authors report that Qwen-3.8-Max found the same four products using nine turns and 25 calls in English, versus 64 turns and 201 calls in Telugu. This is one task, not an average or a general measure of language-specific slowdown.
The paper studies differences in errors, language consistency and resource use across tasks. The reported findings are the benchmark authors’ results, not an independent replication. The paper and dataset README also give different total task counts, so a single count would obscure that discrepancy.
Source: BabelArena authors ↗
Why does it matter?
Multilingual quality is not just a matter of whether an agent eventually returns an answer. The steps, costs and failure patterns along the way can matter to people building services for users in different languages.
Read the sources
- BabelArena research paper ↗
BabelArena authors · October 23, 2026
