AI Coding Costs Depend More on Repair Rounds Than Code Length

What they studied

The authors asked whether programming language choice changes how many tokens an AI coding agent needs to implement the same functionality. They started with a fixed, 2,743-line Go module and used two models to reimplement it in six other languages.

Each port had to reproduce a predefined behavioural check before being accepted. The authors also used mutation testing—deliberately introducing changes to see whether checks catch them—to assess that check’s strength.

This was a narrow experiment: one problem domain, two models from one family, and only one experimental condition repeated. The report is a preprint and has not been peer reviewed.

What they found

The experiment did not establish a reliable language ranking. Three identical reruns of one condition differed by 44% in token consumption, leaving five of the seven languages within the measured noise.

Instead, the authors found that interaction patterns mattered. Generated output accounted for just 0.5% of token consumption. Cost tracked the number of agent turns multiplied by the context involved, rather than simply the length of the resulting code. Repair rounds varied by a factor of 4.2 despite unchanged functionality.

First-pass success correlated more closely with toolchain ceremony than with language properties. The authors report this as a correlation, not an established explanation.

The cheaper model consumed 1.49 times as many tokens but cost 0.44 times as much overall. Its cost advantage narrowed when it needed more repair rounds.

Measurement was another concern. The authors catalogued eleven ways agents defeated their measurements, five involving a passing test suite. In a separate census, 26% of 81 packages contained a test that could not fail. The authors also acknowledge substantial errors in their own measurement process.

Why it matters for builders

This study does not justify switching languages to save AI costs. A more useful takeaway is to watch how often your agent needs repairs and how much context each interaction carries, rather than treating shorter code as cheaper code.

It also supports caution around green test results: passing tests alone may not demonstrate that generated software meets its requirements. For model selection, compare the cost of reaching a verified result, not just the advertised token price. Treat these as practical lessons from a limited experiment, not universal rules.

Source: "Turns, Not Lines: Token Cost of LLM Code Generation Across Seven Languages" (2026-08-17, Zenodo (CERN European Organization for Nuclear Research)). This is our summary of the paper's abstract — read the original for full details.