Repeated reads are real load
A coding agent can list pull requests, inspect one, check its status, and ask again after another tool call. Several agents can make the same request at once. GitHub's REST guidance says to avoid concurrent requests, use conditional requests, keep requests stable, and stop or back off when the service returns a rate limit response.
Ghx takes a local approach. The project puts a daemon between callers and the GitHub CLI. It classifies known read commands as cacheable, coalesces matching requests that are already in flight, and keeps their output in memory for a default 30 seconds. Its project page reports a 1,129 millisecond direct call and a 105 millisecond cache hit for one pull request listing. That is the author's measurement, not an independent benchmark.
The speedup is easy to see. The harder question is whether two calls are allowed to share one answer.
Read the key before the speed chart
We inspected the current main revision c7719e5e4542e1c0a27f71018e95c1d733ab34f4. In src/internal/context/resolve.go, the cache key contains the GitHub host, repository, current branch, a SHA-256 fingerprint of the active token, and the complete argument list. That separates repositories, branches, hosts, accounts, and command forms.
The key does not contain a general copy of the environment, the GitHub CLI version, or file contents. The process forwards named authentication variables to the miss execution path, but those forwarded values are not all represented as independent key fields. The API classifier refuses caching when --input or an uploaded field can add body bytes that are absent from the argument list. Unknown API options also take the mutation route.
This is the useful habit to steal. Write down what the miss can observe. Compare that list with the key. Every omitted input is a claim that it cannot change the answer during the entry's lifetime.
Coalesce the question, not the side effect
Ghx keeps a map of in-flight cache keys. When matching reads arrive together, one executes and the others wait for its result. That is a good fit for repeated observation. It is a bad fit for state changes because a caller needs the effect and its own receipt, not a replayed stdout block.
The classifier sends known mutations down a direct path. The client runs an interactive mutation in the invoking process, then asks the daemon to invalidate the matching resource namespace. The invalidation request still runs after a failed or interrupted command because a partial remote change may have happened. If invalidation fails, the client prints a warning that tells the operator to flush the cache before reading again.
The custom allowlist deserves the same care as executable policy. The source checks user-added cacheable commands before the built-in never-cache list. That makes the override useful, but it also means an operator can choose to cache a command that the default policy would send direct. Custom entries need a source review and an invalidation test.
Thirty seconds is not the policy
A time-to-live limits how long an entry can survive. It does not prove that the output stays correct until expiry. Ghx also invalidates entries by host, repository, and resource after known mutations. The two controls answer different questions. Expiry bounds unattended staleness. Invalidation reacts to a change the wrapper recognizes.
Changes can still arrive outside the wrapper. A teammate can merge in the browser. A workflow can finish. A webhook can update a resource. Another process can use gh directly. A short time-to-live reduces the window, but a release gate should still request fresh evidence with caching bypassed or flushed.
GitHub's conditional request mechanism solves a related problem at the server boundary. An authenticated GET can send an etag back in if-none-match. If the representation has not changed, GitHub can return 304 Not Modified. A local time cache and a server validator are not substitutes. One avoids a request. The other asks the source whether saved data is still current.
Use cache pressure as an observable
Ghx exposes cache hit, miss, coalescing, invalidation, retained byte, and rejection data through its local dashboard and JSON routes. That makes the shortcut inspectable. Watch for a high miss rate, a command with a long override, invalidation warnings, and repeated bypasses near release work.
The implementation caps retained stdout and stderr at 64 MiB by default and limits the entry count to 1,000. It stores stdout, stderr, exit code, cache time, time-to-live, resource type, host, and repository. Cached failures can therefore repeat during the entry lifetime. That may be correct for a stable failure. It may also hide a fast recovery. Include failure output in the freshness policy.
Make the release lane boring
- Use cached reads for exploration, dashboards, and repeated agent context gathering.
- Send writes straight to the real CLI and retain their exit status.
- Flush or bypass before a merge, release, permission check, or incident decision.
- Test every custom cache entry against a mutation and an out-of-band change.
- Record the key fields, time-to-live, storage scope, invalidation route, and owner.
The bench below drafts that record. It does not configure Ghx, call GitHub, or verify a cache under load.