Pimp My IDE / garage dispatchBack to dispatches
October 12, 2026 / coding agents / context cost

The early log stays hot.

A large tool result near the start of an agent run can ride through many later requests. Cut the model copy. Keep the full log and exit status.

Do not optimize for the shortest transcript. Optimize for the smallest context that still carries the failure, the evidence, and the route back to the full result.

A tool result has a position

Agent context is ordered. OpenAI documents prompt caching as prefix reuse. Stable instructions and earlier conversation turns can be reused when the prefix stays the same. Anthropic documents the same basic ordering across tools, system instructions, and messages, with cache reads and writes reported separately.[2][3]

That gives every test log, file dump, and search result two dimensions. Size matters. Entry time matters too. A six-thousand-token log added near the start of a twenty-call run can remain in the input for fifteen later calls unless compaction or another context policy removes it.

The expensive log is the one that enters early and stays hot.

Tokenleak turns the idea into suspects

Tokenleak is a new MIT-licensed Python project for Claude Code and Codex transcripts. Its audit labels large reads, noisy shell output, repeated reads, broad searches, and other patterns. The implementation estimates tool-result tokens at four characters each. It then estimates avoidable cost with provider-specific cache weights and the number of later calls in the same pre-compaction segment.[1]

The project calls these findings estimates, not verdicts. That boundary is correct. A large compiler error can be necessary. A short tail can omit the first failing assertion. Transcript formats change. Cache behavior and prices vary by provider and model.

We exercised the fixture, not the sales pitch

We inspected revision c88e195, release 1.5.1. The audit is one standard-library Python file. Its tests build synthetic Claude Code and Codex transcripts with large reads, repeated reads, noisy commands, compaction boundaries, and malformed inputs. The repository's documented command ran 25 tests on Python 3.11 and returned OK.

That result proves only that the included synthetic cases matched the checker at the inspected revision. We did not install the plugin, attach its hooks to an agent, read private transcripts, or reproduce the project's benchmark. Its reported cost reductions remain author-run results.

Keep the failure outside the model copy

A bounded command should preserve three things. Keep the command's exit status. Keep the complete output at a named path. Print a small tail or focused match into model context. Do not pipe the only run into tail and lose the real status.

run_id=$(date -u +%Y%m%dT%H%M%SZ)
log=".agent-logs/test-$run_id.log"
mkdir -p .agent-logs
set +e
npm test --silent >"$log" 2>&1
status=$?
set -e
printf 'exit=%s full_log=%s\n' "$status" "$log"
tail -n 80 "$log"
exit "$status"

The model sees the status, path, and recent output. A reviewer can open the full log if the tail is ambiguous. Put the log directory in the project's ignore file if it is local evidence. Set a retention rule if the output can contain secrets or user data.

Measure before and after

  1. Choose representative work, not one polished demo.
  2. Record input, cached input, cache writes, output, calls, wall time, and task result.
  3. Change one output rule at a time.
  4. Replay the same acceptance check.
  5. Inspect misses. A cheaper wrong answer is still wrong.
  6. Keep the rule only when repeated runs preserve the result.

The dyno below models repeated input volume. It does not calculate a bill. Provider caching, compaction, hidden instructions, and model pricing remain outside the fixture.

Interactive teaching fixture

Context heat-soak dyno

Traditional purpose: estimate repeated context from a tool result. This control joins result size, entry time, output policy, evidence checks, and a copyable command brief.

CaptureStoreBoundInspect

Heat inputs

The session has twenty model calls. Set the result size and the call that produced it. Choose what later calls receive.

6,000 tokens
1,000 tokens12,000 tokens
5 of 20
Call 1Call 19
Later-call output policy
Evidence fields selected
RECIPE INCOMPLETE2 of 4 evidence fields selected

Repeated input preview

The raw fixture remains available to fifteen later calls. This is repeated input volume, not a provider bill.

15 later callsRaw result90,000 token placements
RAW RESULT STAYS HOTThe complete fixture remains in later context. No reduction is modeled.
Sources read / October 12, 2026 UTC

Source log

Open the evidence boundary
  1. Tokenleak repository at commit c88e195, version 1.5.1, read October 12, 2026. We inspected the README, changelog, package manifest, audit implementation, and tests. We ran python3 -m unittest discover -s tests against the exact checkout. It ran 25 tests in 0.160 seconds and returned OK. We did not install the package or hooks, read personal transcripts, or reproduce its benchmark.
  2. OpenAI, "Prompt caching", read October 12, 2026. It supports the stable-prefix model, cached-token accounting, cache-write accounting, and the warning that compaction can reduce cache reuse.
  3. Anthropic, "Prompt caching", read October 12, 2026. It supports prefix ordering, cache reads and writes, automatic and explicit breakpoints, time limits, and provider-reported usage fields.
  4. Hacker News item 50055322, resolved through the official API and read October 12, 2026. It was the discovery route. It supports none of the implementation or benchmark claims.

Evidence boundary: the 25 passing repository tests cover included synthetic fixtures. They do not prove transcript compatibility with every agent release, the accuracy of four-characters-per-token estimates, the correctness of every finding, or the project's reported cost reductions. The dyno uses invented values and does not calculate provider charges.