Automation eats the apprenticeship.
Sylvain Kalache’s warning lands because it is not the usual “AI bad” foghorn. Routine incidents are useful. They are where an on-call engineer learns which graph lies, which cache is load-bearing, who can approve the ugly mitigation, and how a system sounds before it breaks. If an automated responder quietly closes those cases, the wake-up page is increasingly likely to be the ambiguous one it could not solve.
That shape is older than LLMs. Lisanne Bainbridge’s 1983 paper Ironies of Automation argued that removing operators from routine control can make the remaining manual task harder: people are expected to take over abnormal situations while deprived of the practice and system contact that build skill. Kalache applies that human-factors problem to modern incident response and argues for simulators, not nostalgia.
Keep the robot on night shift. Make its receipts feed the flight simulator.
Explanation is a receipt, not a rep.
An agent should absolutely leave evidence: alerts inspected, queries run, hypotheses rejected, change made, and rollback condition. That makes review possible. It does not reproduce the work of forming a hypothesis with incomplete information while coordinating people under pressure.
Google’s SRE workbook makes the social machinery explicit. Incident management is not only resolving the technical fault; it is coordinating response and keeping communication flowing. Its core practices include a clear command line, defined roles, a working record, and early declaration. Those behaviors need rehearsal before the graph turns red. Reading a perfect machine postmortem afterward is closer to watching game tape than taking the snap.
Measure contact, not theater.
The lazy fix is a quarterly tabletop where everyone nods at a slide. The better drill removes the answer key, gives the commander live but incomplete signals, and requires an actual mitigation, rollback decision, status update, and evidence log. Reuse a sanitized real incident so the failure shape belongs to your system instead of a generic “database is slow” screenplay.
Do not turn the drill score into fake production telemetry. Count exact things: incidents closed by automation, human-led cases, completed simulations, roles rotated, and days since the last hands-on rep. Treat any readiness dial as a teaching proxy. A team can complete four drills and still freeze on a novel cascade.
The garage policy.
- Attach a drill seed to every automated close. Preserve the initial symptoms and timeline separately from the diagnosis.
- Run blind before replay. Let responders investigate without the machine’s final answer, then compare paths.
- Rotate command and communications. Debugging skill does not substitute for coordination under load.
- Practice the handback. Trigger the exact point where automation stalls, escalates, or loses authority.
- Track recency. A runbook last touched twelve months ago is a museum label.
The Hacker News discussion around Kalache’s piece is useful as shop-floor reaction, not controlled evidence. Commenters describe patient isolation being displaced by confident shotgun fixes, but others report measurably better model-written code. Keep both. The point is not that automation always weakens people. The point is that capability decay is plausible enough to instrument and rehearse instead of discovering it during a SEV0.