Software factory

The lights go out last

“Software factory” is the 2026 name for a team where agents write the code and nobody reads it. The phrase is easy to say and easy to fake. One question separates the real ones from the reckless ones, and it is the only thing to learn tonight: when the human stopped reading the diff, what started doing that job?

1. 1 · SeedA spec. The human writes it.
2. 2 · ValidateA check the builder cannot game.
3. 3 · Feed backFailures return. Loop until it holds.

StrongDM’s own summary: “Seed → Validation harness → Feedback loop. Tokens are the fuel.” Principles

Dan Shapiro’s ladder, modelled on the levels of driving automation. Read the right-hand column. The climb is a human check being removed, one rung at a time.

Where the name came from

In January 2026 Dan Shapiro published five levels of AI-assisted coding, from “spicy autocomplete” to the “dark factory”, named after factories that run with the lights off because robots do not need to see. He puts most developers around level 2 and calls level 5 “a black box that turns specs into software,” run by teams of fewer than five. The Five Levels · Simon Willison’s notes

A week later StrongDM’s three-person AI team described working that way since July 2025, under two rules: “Code must not be written by humans” and “Code must not be reviewed by humans.” Willison on StrongDM

You already own every part. The loop is 0012, where the maker is not the checker. The check is 0014 and 0019. The wall around it is 0020. A factory is those three running with no one watching.

• Seed “Every piece of software needs an initial seed”: a spec, a PRD, a screenshot, an existing codebase. The human’s work moved here, to the front.
• Validate End to end, “as close to the real environment as possible.” StrongDM keeps its scenarios outside the codebase, like a holdout set, so the builder cannot see what it will be graded on.
• Feed back Output is sampled and fed back in, so the system can “self-correct and compound correctness.” It stops when the holdout scenarios pass “and stay passing.”

Architecture first. The human did not leave. They moved from the middle of the loop to its two ends.

The question that sorts the real from the fake

Removing review is the easy half. Anyone can stop reading diffs. Simon Willison, who visited the StrongDM team, names the hard half: “how can you prove that software you are producing works if both the implementation and the tests are being written for you by coding agents?” Agent-written tests only help “if they don’t cheat and assert true.” Willison

Anthropic found the same thing from the other side: asked to grade their own work, agents respond by “confidently praising the work — even when, to a human observer, the quality is obviously mediocre.” Harness design for long-running apps

So the test for any rung is one sentence: name the thing that replaced your eyes, and say why the builder cannot fool it. If you cannot, the lights are off and nothing is watching.

# Same billing assistant as 0014–0020: HTTP + DB + Stripe.

LEVEL 3 · you read the diff
  you     prompt: "add proration to the refund policy"
  agent   edits policy.ts and refunds.test.ts, opens a PR
  you     read 240 lines, ask for two changes     # your eyes are the check
  you     merge

LEVEL 5 · factory
  SEED      you    specs/refund-proration.md      # intent + acceptance criteria
  VALIDATE  you    scenarios/annual-midyear.md    # kept where the builder cannot read
            twin   a fake Stripe: refunds.create returns re_ ids, records amounts
  LOOP      agent  builds
            check  runs every scenario against the twin        6 of 9
            agent  receives the three failures, rebuilds       8 of 9
            agent  again                                       9 of 9, and stays 9
  SHIP      nobody opened policy.ts

  you read: the scenario board, one trace (0018), the twin's refund ledger

FAKE LEVEL 5
  agent   writes policy.ts AND refunds.test.ts, tests green, auto-merge
  check   none that the builder did not write itself

Skill: name what replaced the eyes

Six teams. For each, decide what stands between the agent and production. Answer before you look for the reason.

Three field reports, and the bill

Each report is written by the team that did the work, about a project started from nothing. Read them as evidence that it can be done, and then read the costs they and others state:

• Money. Willison on the $1,000 a day: if these patterns add $20,000 a month per engineer, “they’re far less interesting to me.” He hopes for a version at far lower spend. That version is what 0025 is about.
• Transfer. OpenAI: the behaviour “depends heavily on the specific structure and tooling of this repository and should not be assumed to generalize without similar investment.” All three were new codebases.
• False green. The compiler’s author: “it is easy to see tests pass and assume the job is done, when this is rarely the case,” and deploying software nobody has personally verified “is a real concern.”
• Accountability. A Stanford Law commentary on StrongDM: when agents write and agents test, “nobody knows who is liable,” and contracts still assume a human reviewed the code. Built by Agents, Tested by Agents, Trusted by Whom?

What to do with this on Monday

Parked for the next three lessons: holdout scenarios and twins (0023); the repository as the system of record, a short AGENTS.md as a map, and rules promoted into linters (0024); the ramp, the budget and where to keep a human (0025). StrongDM’s named techniques (Attractor, gene transfusion, semports, pyramid summaries) are parked until one of them is the next thing you need.

Check