Pimp My IDE / Garage Dispatch
Back to garage
September 20, 2026 | agent loops / goals / evaluation

A software factory still needs a foreman.

The useful part of an agent factory is not endless motion. It is a loop that can read a goal, inspect the gauge, choose work, and stop when the evidence changes.

THE TAKE: Do not hand an agent a broad objective and call the loop autonomous. Give it a measured target, one visible queue, a bounded work policy, and a stop condition that a human can inspect.
Walk the Factory Foreman Rail

The loop starts above the ticket.

Will Larson describes an early software-factory loop at Imprint that reads a Linear project, checks whether the goal and measurement exist, reviews current metrics and issues, adds missing work, and then acts on unblocked tasks. When the project description grows stale, the loop returns to the top instead of pulling another ticket.[1]

That detail separates a project loop from a task conveyor. The agent does not only ask, "What is next?" It also asks whether the queue still represents the goal.

A factory claim needs its own boundary.

StrongDM describes a more aggressive model. Its software factory uses specifications and scenarios to drive agents that write code, run harnesses, and converge without human code review. The team says tests alone were easy for agents to satisfy narrowly, so it added scenarios outside the codebase and behavioral copies of third-party services.[2]

Those are StrongDM's reported practices and results. They do not establish that every project should remove review or build service copies. They do support one hard lesson. A loop needs an evaluation environment that the worker cannot quietly redefine.

The queue tells the factory what to do. The gauge tells it whether the work mattered.

Stop polishing the instruction in isolation.

Mike McFunley argues that teams should treat prompts as implementation details and put domain effort into datasets of good and bad responses, deterministic assertions where possible, and usage monitoring that exposes failures. His examples focus on conversational agents, but the same division helps coding loops. The durable asset is the behavior test and its labeled evidence, not one lucky paragraph.[3]

A software factory can still use prompts, skills, and project descriptions. Keep them versioned. Judge them against a fixed scenario set and observed outcomes. Do not let the same worker edit the target, the gauge, and the verdict in one pass.

Give the loop four stations.

  1. Objective: name one outcome, owner, scope, and non-goal. A theme such as "improve reliability" is not a target.
  2. Gauge: point to the query, dashboard, test, or labeled scenario set that can show movement. Record its baseline and failure modes.
  3. Queue: use one visible source of work. State who may add, reorder, close, or reopen items.
  4. Stop: cap time, spend, retries, change size, and authority. Define what returns control to a person.

The loop can then propose work, test it, and update the queue without pretending motion equals progress. A human still owns the objective and any decision that expands the factory's reach.

Interactive makeover / loop contract

Factory Foreman Rail.

Traditional purpose replaced: paste a broad goal into a recurring agent job. Better version: choose the loop cadence, close four operating stations, and copy a contract that keeps every blank evidence field visible.

Put the work behind an external gauge

This teaching rig does not inspect a project, metric, queue, or runtime. Its status comes only from the controls below.

Loop cadence
Operating stations
BLUEPRINT ONLY0 / 4 STATIONS

The line has no operating contract.

Choose a cadence and close the stations with real project evidence before the loop receives work.

Why it is better: cadence, objective, measurement, work selection, and shutdown share one state. The final setting says "review gate ready" because the rig never reads the project or fills the evidence fields.
Sources read, not vibes
  1. Will Larson, "Trying the Software factory pattern": Imprint's reported project loop, goal and measurement audit, issue updates, task selection, stale-description reset, and dependencies on Linear, Datadog, Snowflake, and an orchestrated harness.
  2. Justin McCarthy, "Software Factories And The Agentic Moment": StrongDM's account of specification-driven development, scenarios, satisfaction scoring, behavioral service copies, and its no-human-code-review claim.
  3. Mike McFunley, "Prompts aren't Real": deterministic assertions, labeled good and bad interactions, usage monitoring, and the argument that domain experts should build evaluation artifacts instead of curating prompts.
  4. Hacker News discussion, item 49777913: the exact discovery thread for Larson's article. It is a reaction source, not proof that the pattern works.

Source boundary: the three authors describe different systems and opinions. Pimp My IDE combined them into the four-station control rail. It is a planning aid, not production telemetry, a safety certification, or proof of business impact.