Prediction leaves a compression receipt.
The paper "Language Modeling Is Compression" starts from an established equivalence. A probabilistic model can drive a lossless compressor, and a lossless compressor implies a prediction model. The paper also shows how a compressor such as gzip can act as a conditional generator.[1]
Nathan Barry turned that last idea into GziPT. The program primes zlib with a text corpus, tests possible byte continuations, and keeps candidates that produce shorter compressed output. Beam search helps because gzip reports whole-byte lengths. Many one-byte candidates otherwise tie.[2]
The output looks a little like Shakespeare because the corpus contains Shakespeare. It also breaks down quickly. That failure is useful. It makes the shortcut visible.
Gzip has a short rear-view mirror.
DEFLATE combines LZ77 back-references with Huffman coding. RFC 1951 limits match distance to 32 KiB and match length to 258 bytes.[3] GziPT exploits that local match machinery. A continuation costs less when it resembles bytes still visible in the window.
Local repetition is not repository understanding. A coding assistant can also look capable when the target pattern sits nearby, names stay unchanged, or the benchmark asks for a familiar patch. Move the evidence and the score may collapse.
A baseline is useful when it wins for the wrong reason. It tells you which shortcut the real system must beat.
Test the shortcut, not the brand.
Start with a task that has an executable check. Ask for a bug fix, migration, or narrow feature. Record the result with normal repository context. Then rerun matched variants that remove one cheap route at a time.
- Hold out the target file until the task begins.
- Move the relevant example beyond the assistant's immediate context.
- Rename local symbols without changing behavior.
- Add a nearby pattern that looks right but fails the test.
- Keep the task, budget, tools, and checker fixed.
If performance survives, you learned more than "the demo worked." If it fails, you found a dependency on proximity, naming, or copying. That is a result worth keeping.
Do not confuse compression with comprehension.
The compression paper reports strong results for large models, but it also says model parameters must count toward adjusted compression rate. Larger models can become worse compressors once their own size enters the bill.[1]
The same accounting belongs in agent evaluation. Count repository indexing, prompt assembly, retries, tool calls, review, and the checker. A polished patch with hidden setup costs is not a clean comparison. A short output that copied a nearby template is not proof of a useful internal model.