Software Engineering is going through the biggest change I ever experienced.

I keep hearing about dark factories. Software written by agents, verified and merged all automatically. I read about that some people are more often saying that the question is rather not if but when this is coming. Software will be cattle rather than pet.

Other developers and I are adopting spec driven development in one way or another. This is one part of the journey.

I see two areas to tackle next.

How do the requirements from product get into a form that might be usable by development?
How do we figure out that what got developed is what we want, in the shape and form we expect.

Let’s look at the latter one. How can this look like? A bit deeper look how this lifecycle can be.

Requirements

Imagine a product manager writes up his PRD (maybe with some skills) and refines that with developers. The resulting tickets need to have some acceptance criteria which an agent can develop against. It is written in a way that an agent can figure out the proper order of the tickets if they have dependencies.

This gets added to a ticket system with a label ‘factory’.

Software

The repository needs to have some prerequisites.

Build & run

One way to run the software and spin up everything which is needed. This also includes mocks and everything you need, depends on your isolation for tests, like make bootstrap. A make verify can make sure everything passes basic tests.

Context

A standard way where the agent can find enough context, like a CONTEXT.md, ARCHITECTURE.md and ADRs. An AGENTS.md or CLAUDE.md helps guiding the agent.

Orchestrator

The orchestrator runs and checks regularly if there are tickets with the factory label. If it finds one:

That is the next one to be implemented without dependencies.
Takes the ticket and creates a prompt out of it with instructions for the agent to implement -> prompt.md
Spins up a docker image fitting for that particular repository
Checks out the code
Runs make bootstrap
Runs the agent with prompt.md
Agent implements and runs make verify
Only if that passes it creates a branch and pushes it, leaves metadata in the workdir, branch name, screenshots etc
Orchestrator takes branch name out of metadata and creates PR with things like screenshot etc as proof as a comment

Necessary metadata and tokens are passed to the docker image via env vars.

The make verify before the push is important. If it fails, nothing gets pushed. The container is thrown away and the ticket goes back in the queue for another try. Failed work is never going to be a PR.

The workflow will run the same checks again later on the PR, but by then it is a second opinion (fresh agent).

The agent can also stop and ask. If the ticket is ambiguous, or it would have to break something to do it properly, it posts one question on the ticket and stops instead of guessing. This way a blocked ticket can be seen and refined. An agent guessing with confidence costs more down the road.

Verification

Once the PR is opened a workflow is triggered to verify the results. Every check reports back as a commit status on the PR, one per check.

A commit status is a small record on a commit - a name, pass or fail, and one line of text. Every check picks its own name, like ci or verify/no-cross-module-imports. Branch protection config on main holds a list of names that have to be green before anything can merge, and you can use patterns - verify/* covers every verifier, so a new one becomes required the moment you write it.

Only the agentic review in tier 5 leaves real comments on the diff.

Listening to podcast, reading about this topic and forming the tier ladder below, was the change for me to understand it better, to get the impression this might work. More deterministic checks which I can trust.

There are different types of tests:

Tier 1 — Deterministic checks (milliseconds, free)

Formatters, linters, type checkers, dependency-boundary rules, schema validation.

Anything you can make deterministic, make deterministic.

Tier 2 - Tools like SonarQube (minutes, free)

Set a quality gate condition on coverage of new code. It is the gate that will stop an agent shipping untested code, and it is deterministic. Make it boolean decision.

Tier 3 - Tests (seconds to minutes, free)

Unit, integration, end-to-end.

Test behaviour, not implementation. The factory will rewrite implementations freely. Tests coupled to internals can produce constant false failures. Tests coupled to observable behaviour are the contract the factory works against.
Your end-to-end tests are your real specification.

Tier 4 — Verifiers (one cheap LLM call each)

A verifier is a single natural-language assertion about a diff that resolves to a boolean, evaluated by one LLM call, judging only that one thing.

A definition could look like:

# .factory/verifiers/imports.yml
- id: no-cross-module-imports
  paths: ["apps/api/**/*.py"]
  assert: >
    No file imports from another application module directly.
    All shared code is imported from lib/.
  severity: block

- id: migrations-are-reversible
  paths: ["db/migrations/**"]
  assert: >
    Every migration defines both an up and a down path, and the down
    path fully reverses the up path.
  severity: block

- id: no-silent-except
  paths: ["**/*.py"]
  assert: >
    No exception handler swallows an exception without either logging
    it or re-raising.
  severity: warn

This is better than a big review prompt:

Deterministic triggering. The path glob decides whether it runs. No “the reviewer didn’t notice.”
One concern per call. A review agent holding fifteen competing priorities will drop some of them. A model asked one yes/no question about one diff answers reliably.
Cheap and parallel. Each is a single call on the diff. Ten verifiers cost less than one deep review.
Human-understandable. You can read a verifier and know exactly what it does. You cannot read a 400-line review skill and predict how it weighs anything.

Verifiers are how you encode taste, describe how your code should look like. Historically you’d enforce those in code review, in each PR. Now you write it once and every agent obeys it forever.

Tier 5 — Agentic review (expensive, slow, catch-all)

A full review agent reading the diff with repo context. Give it a small number of distinct tasks / focus rather than one generic prompt -> security, maintainability, functional correctness, run each as a separate pass / marker. The job is to catch what the lower tiers missed. Anything it catches repeatedly should be implemented in the earlier tests.

Fixing findings

The orchestrator collects the findings into a prompt. Starts a new container with that branch and the prompt and pushes the changes which it thinks need to be done, make verify runs again.

When done, pushes these changes to the branch and the git workflow will be triggered again.

The orchestrator will update the status / labels during going back and forth. He also checks if one ticket went too often to that loop and can halt it by setting a need human label and remove the factory label.

Merging

Merges happen without humans looking at it. But not everything should.

It is classified by path in a file you can read, e.g.

Docs, tests, tooling, non-user-visible refactors are low risk.
Backend behind existing tests is medium. Auth, payments, migrations, anything a user sees is high.

The riskiest file in the diff decides the whole PR - twenty docs and one migration is a migration.

Only the classes which are nominatated merge on their own. Everything else waits for a human, with a comment saying which rule made that call. I would start with low only and widen when the verifiers for those paths have earned it.

This is kept as a rules file, not something a model decides. When something bad lands (and it will) I want to open a file, read the line that said this was safe, and change it.

Example:

 # .factory/risk.yml
  default: med

  rules:
    - paths: ["docs/**", "**/*.md", "tests/**"]
      risk: low
      why: no runtime effect

    - paths: ["scripts/**"]
      risk: low
      why: dev tooling, does not run in production

    - paths: ["src/**"]
      risk: med
      why: covered by the existing test suite

    - paths: ["src/**/money/**", "src/**/billing/**"]
      risk: high
      why: rounding mistakes are silent and expensive

    - paths: ["src/**/auth/**", "db/migrations/**", ".gitea/workflows/**"]
      risk: high
      why: blast radius is the whole system

The merge itself is not the orchestrators job. Branch protection on main lists the checks that have to be green. The orchestrator asks the git server to merge as soon as all of them pass, and the git server does it when they do. Branch protection with these rules is essantial here.

The number to watch is the percentage of PRs merges without a human. Thats the one KPI.

What this does not tell me

All of that above tells me that the code is correct. None of it checks that it is the thing I asked for.

Nothing in the loop reads my ticket and checks it.

The ticket contains acceptance criterions I can tick off. The agent turns it into a test before it starts implementing. From then on tier 3 runs that test like any other test. And this is really important now, I need to make sure I have the right, not too vague criterion in the ticket. Otherweise, all tests pass, everything is green and the wrong thing shipped.

This brings the question of what is right to the ticket, to the PM, to the developers. Really good tickets with good acceptance criteria, the unicorn in development, are now cruicial. Thankfully, good skills help.

What does that mean for developers

We will see a shift from coding to specification. Engineers need to pivot from writing syntax to defining precise specifications and rigorous testing protocols. A shock to almost everyone. I dont know how many posts on reddit I see about this. Coders and builders.

We need to mitigate cognitive debt risks. to precent skill erosion by maintaining active understanding of system architecture and logic. Still important. Blind reliance on AI outputs creates critical vulnerabilities in understanding / debugging and long-term system stability. We have to up the security game.

And the role change:

Engineering and Product Roles. Developers must cultivate product sense and UX intuition to remain competitive. The convergence of technical and business roles demands a broader skill set than pure coding. This shift redefines the core value proposition of technical talent in the AI era.

How to start

I would / will not start with all of it. The hardest part which will break everything for me is the warm and fuzzy feeling I need to have when an agent runs through the verifier loop. If I do not have that, it’s not worth investing time in it.

Many of the checks you can start developing now, just running locally. Learning what works and what not. I think the verifiers are the most important ones. They need to earn your trust that they do the right thing.

Without all the orchestrator, passing context, env vars back and forth you could start today with

Writing good tickets with ACs
Writing verifiers to learn what works and what not.

Thoughts

This all reminds me of the early 2000s as we introduced CI/CD. We automated away systemadminstrators which deployed software to (on prem) production and database admins who wanted to write migrations by hand because only they knew how to optimize. Servers were configured by hand back then and are now fully automated. Then the cloud came and many moved on.

This now feels like infrastructure work again and a step to automate wirting code in the editor away. We are shifting left. Closer to product, closer to customers.

Best practices have not formed yet, the landscape is still evoloving every day. New models, tooling, skills, practices. Every. Single. Day. Main challange is deterministic vs undeterministic. Neck breaking speed.

We as engineers have to adapt. I can not see this going away. Will it change again? Yes. Do I know what will it be like in 6, even 3 months from now? No, but it will be different.

Embrace the change as much as you can.

I started to build this loop. Not there yet. Complicated and many things to tinker and discover. That’s why I wrote this text to understand it better for myself. I started my career as system administrator.

Thanks for making it through my thoughts, a bit rough on the edges but I needed to write it down.

Hey, this already sounds like a PRD. Maybe I can give it an agent to implement.

Version 1