Agents get measured, tokens get cheaper, output gets longer

This week the field put numbers on two things it usually argues about: how long an agent stays reliable, and what a cheaper token actually costs per task. Both answers point the same way. The unit that matters is the task, not the call. Underneath, a second cluster of papers moved the proof in front of the deployment, and the harness around the model became a research object in its own right.

Agents, measured over long horizons

The most useful paper of the week is also the least surprising once you see the curve. Across nine models from 1.2 to 671 billion parameters, four task families, five task lengths and three context conditions, success follows a geometric law¹01 arXiv 2609.01660 with a single per-step reliability. Benchmarks cover short to medium horizons; production asks for an order of magnitude more dependent steps. That is the gap between a good leaderboard score and an agent that quietly fails on a Thursday.

Two companion pieces make the failure modes concrete. A case study hands a coding agent a complete specification, storage engine, schema, entity resolution and filter strategy all fixed, and catalogues the five errors it still introduced²02 arXiv 2609.01985, ordered by which constraint broke and how the break was found. AGENTSCOPE³03 arXiv 2609.02371 takes the opposite route to debugging: instead of asking a model to judge a long trace, it compresses the trace into a structured representation and checks it against named invariants, because a model as the sole judge produces unreliable findings. A third paper shows why outcome-only grading is not enough: over 400 runs with planted errors⁴04 arXiv 2609.00038, the evaluation most teams use, request plus final answer, was blind to the case where the agent reached the right result by the wrong path.

The agents themselves do not notice any of this. Claude Code and Codex systematically overestimate how long a task will take⁵05 the-decoder.de, Codex by up to a factor of ten, and rate their own work about 20 percentage points too well. One practical answer arrived the same week: estimate from the visible trace of a browser agent whether it is still on course⁶06 arXiv 2609.02057, without access to token probabilities, and flag the first uncorrected step rather than the final result.

The token economics flipped

Google shipped its third Flash model in six weeks. Gemini 3.8 Flash⁷07 the-decoder.de keeps the token price and needs roughly 30 percent more output tokens per task, which makes it more expensive than its three-week-old predecessor on exactly the workloads that matter. The day before, the same pattern was reported for Claude Fable and Mythos 5.1⁸08 latent.space: 75 percent less for cached input, 70 percent more output. GitHub's own write-up on Copilot says the quiet part out loud⁹09 github.blog: shorter answers can cost more in the end, so the cost has to be accounted per task, not per reply. Anthropic's own framing of Fable 5.1 is the same sentence from the other side, up to 45 percent lower cost¹⁰10 the-decoder.de, mostly on long autonomous runs with many tool calls.

The pricing models are following the accounting. OpenAI is charging some large customers only for completed tasks¹¹11 the-decoder.de, with Salesforce, Adobe and several start-ups moving away from flat subscriptions in the same direction, and with the attribution question left open. Access itself became a business risk in the same week: OpenAI ended its model contract with Cursor¹²12 openai.com after the SpaceX acquisition, and Anthropic replaced a temporary 50 percent increase in Claude Code weekly limits with a permanent 25 percent¹³13 the-decoder.de from 14 September. On the research side, a 0.6-billion-parameter router scores each candidate model's fitness for a task¹⁴14 arXiv 2609.02292 without generating an answer, which is the kind of component this pricing environment will need.

Proof before deployment, and a budget for reach

Several groups moved the check in front of the go-live this week. READY¹⁵15 arXiv 2609.02095 qualifies an agent per workflow by required reliability, tolerable oversight effort and cost, and picks the cheapest oversight rule that still holds the threshold, rather than qualifying by benchmark result. Two randomised field experiments with 640 customer-facing employees¹⁶16 arXiv 2609.01976 show that people find errors in AI output when the information needed to check is at hand at the moment of review; self-written justifications improved detection and reminders kept it up on repeated use. On the permissions side, EAL-Bench¹⁷17 arXiv 2609.01836 tests whether an agent's persistent memory correctly tracks granted and revoked permissions, and finds that a wrongly continued entry can create an authority that was never given, without any external attack. In a separate study of 480 runs over MCP and A2A¹⁸18 arXiv 2609.01693, an agent passed fields marked confidential verbatim to a second agent, counted by fixed rules with no model as judge.

Two papers put a number on how far an agent may reach. One keeps a runtime account of irreversible effects per principal¹⁹19 arXiv 2609.00275, because a fleet of individually approved agents can overrun its principal's risk under a shared trigger while every local check stays correct. The other, from earlier in the week, moves approval out of the agent's reasoning and to the tool boundary²⁰20 arXiv 2608.27646: an agent with a human's credentials inherits the human's reach without the human's judgement, so the enforcement narrows the query before the call and filters the response. The commercial version of the same trade appeared on Thursday: Anthropic offers enterprise customers zero data retention again²¹21 heise.de, on condition that the customer takes over abuse monitoring.

The harness becomes the object of study

For a long time the interesting variable was the model. This week five papers treated the scaffolding around it as the thing to measure. A survey of 1,926 repositories with Claude Code plugins²²22 arXiv 2608.28497 asks whether instruction files, scripts and configuration are maintained together or written once and left; SpecMine²³23 arXiv 2608.25202 counts 470,795 specification files in 73,030 repositories to see how spec-driven development actually spreads. Two failure studies are the ones to keep: a forced structured-output setting silently disabled tool use²⁴24 arXiv 2608.28439, so the model passed a consistency check without ever opening the datasheet and returned invented code, visible only in the per-call tool log; and on real Kubernetes manifests, a lenient patch tool applied 96 percent of agent diffs²⁵25 arXiv 2609.00227, of which 14 to 20 percent landed wrong without any error message, while strict application accepted almost none.

Two design papers propose fixes at the harness level. When an agent optimises its own scaffold, later calls see results but not the assumption behind the previous change, so one paper keeps that hypothesis as its own document²⁶26 arXiv 2609.01861. SkillGLoW²⁷27 arXiv 2609.02217 consolidates skills written from individual runs into procedure families detached from the triggering case, because a single collection document gets too general and a flat pile of entries stays stuck to its origin.

The practice reports are less comfortable. Paint.NET now ships a 180,000-line Direct2D reimplementation written by Claude²⁸28 simonwillison.net that nobody has reviewed in full, because 180,000 lines cannot be read; the rest of the application is about 70,000 lines. Coding agents from Claude, Qwen and Grok ran malicious code on opening a prepared repository²⁹29 heise.de, with the user's full rights and no interaction. And the maintainer of the OCaml compiler reports that a rumour of a bug is now enough³⁰30 simonwillison.net: about ten minutes after a patch was discussed publicly, a server saw requests with encoded path-traversal sequences. The time from shared patch to exploitation has shrunk from days to minutes.

MCP is rebuilt at both ends

The Model Context Protocol revision of 28 July requires stateless communication³¹31 heise.de, the deepest change since the first release; anyone building on it now has to structure the application differently from the start. In the same days Anthropic extended the pattern to hardware with a Model Hardware Standard³²32 heise.de for lab instruments, robot arms and lasers. The clients followed: Codex 0.152³³33 github · openai/codex allows package names as MCP server names and an output-token limit per tool, and Claude Code 2.1.259³⁴34 github · anthropics/claude-code lets an organisation push HTTP and SSE servers to every user through a managed setting.

Papers worth your time

• Separate the leader from the executor. LoopArena³⁵35 arXiv 2608.28281 puts a model in charge of deciding what a separate coding agent does next, and measures only that control loop, because a failed run cannot tell you whether guidance or execution broke.
• Benchmarks are written like tickets, requests are not. 87 percent of real requests are casual³⁶36 arXiv 2608.27831 while 94 percent of benchmark tasks are formally written, which is a good reason to distrust any coding-agent score on your own backlog.
• Quantization damage is not where the rules of thumb put it. Measured per layer across nine open models from four families³⁷37 arXiv 2609.01587, neither task circuits nor weight statistics predict which layers need 8 bits back, and in eight of nine models about half of all layers are needed to recover 75 percent of the loss.
• The model names where to look. Declarative Attention³⁸38 arXiv 2609.02737 lets the model state in its reasoning which parts of the context it needs, instead of scanning the full key-value store for every token.
• World models for browser agents, trained to discriminate. Instead of predicting the next page as HTML, the objective is to tell the reached state apart from the states other actions would have reached³⁹39 arXiv 2609.02885, on a branching dataset built from WebArena.

Rising repos

• 40 github · Pinvou/pinvou-agent
  Pinvou/pinvou-agent⁴⁰+741,589 stars

  an open-source agent for the desktop that combines tools, files and applications on the local machine.

• 41 github · duty1g/x64dbg-mcp-server
  duty1g/x64dbg-mcp-server⁴¹+611,849 stars

  exposes the x64dbg debugger fully over MCP, so an agent can set breakpoints and read memory.

• 42 github · yjh051108/dsh-routing-suite
  yjh051108/dsh-routing-suite⁴²+537,034 stars

  a routing layer inside the DeepSeek Harness environment that dispatches tasks across models.

• 43 github · zorost/AI-Engineering-Lab
  zorost/AI-Engineering-Lab⁴³+73306 stars

  a 24-week course with 43 runnable notebooks along one continuous project.