the harness is not enough david abram the harness is not enough why context engineering matters ai makes code generation fast. it does not make software evolution safe. 2 / 25 ai made implementation cheap • agents implement changes in minutes • review, alignment and maintenance still take human time • more generated code means more decisions entering the codebase code generation is no longer the bottleneck. coordination and human reason are. 3 / 25 introducing sequence of loops Intent
↓
Planning
↓
Architecture
↓
Implementation
↓
Verification
↓
Review
↓
Production feedback
↺
the agent only replaces part of the system 4 / 25 the tempting conclusion when review becomes the bottleneck, the tempting response is: ▍ automate the review and remove the human
that produces a dark factory — lights off, no humans on the floor:
• agents implement • agents test • agents review • agents fix incidents • nobody reads the code 5 / 25 why more loops don't save you more agent loops catch more mistakes. but every agent in the loop shares the same model limitations. they do not guarantee architecture or maintainability over time. 6 / 25 passing today ≠ maintainable tomorrow a change can be: • correct now • fully tested • accepted by ci • clean code tm ...and still make the next change harder. 7 / 25 the feedback asymmetry Correctness feedback: seconds or minutes
Architecture feedback: weeks or months
there is no fast, reliable way to measure: ▍ will this design remain easy to modify after twenty more requirements?
models are trained against the signal we can measure quickly 8 / 25 long-horizon benchmarks expose the problem traditional benchmark: Here is the complete issue.
Fix it.
Run the tests.
long-horizon benchmark: Checkpoint 1 → implement
Checkpoint 2 → evolve
Checkpoint 3 → evolve again
...
All previous behaviour must still work.
▍ can the model leave the codebase in a state where the next task is still
▍ easy and safe?
9 / 25 frontier models still accumulate defects SlopCodeBench benchmark paper: • 20 evolving software problems • 93 total checkpoints • 11 coding agents evaluated • each agent extended its own previous implementation • earlier requirements remained under test no agent completed a single problem end-to-end highest checkpoint solve rate: 17.2%
quality degraded across the trajectories: • structural erosion increased in 80%
• verbosity increased in 89.8%
Source: SlopCodeBench: Benchmarking How Coding Agents Degrade Over
Long-Horizon Iterative Tasks, 2026. https://arxiv.org/abs/2603.24755
10 / 25 the familiar production pattern 1. the first generated feature looks impressive 2. the next few additions still feel manageable 3. complexity and exceptions accumulate 4. new requirements fight the original structure 5. the agent patches around previous decisions 6. nobody understands the whole trajectory 11 / 25 metrics explain, they don't decide • a small function is not automatically good • a large function is not automatically bad • duplication may be temporary or intentional • lint rules reward code that looks clean while staying conceptually wrong the strongest signal is still behavioural: ▍ can the system keep accepting new requirements without breaking old ones?
12 / 25 the answer is not to stop using agents ▍ 100x faster with nobody reading the code
NO! ▍ 2-3x faster writing code while preserving human-level engineering
YES! 13 / 25 the four control points review at the end is expensive: every implicit decision is already encoded in thousands of lines. product intent — the problem, who has it, what success means, what we are
not building
system architecture — services, boundaries, schemas, apis, data flow,
ownership, rollout program design — types, signatures, module boundaries, file layout, call
graphs, failure behaviour vertical slices — small end-to-end increments you can exercise and review
before continuing review the decisions before reviewing their consequences 14 / 25 the missing layer is shared, durable context Human intent Human intent
↓ ↓
Conversation Versioned shared context
↓ ↓
Agent guess Agent execution
↓ ↓
Code Evidence and trace
↓
Code and future context
on the left, every session re-derives how the repository is structured, which constraints matter, why previous decisions were made, what was rejected, and what "done" means. reviewers then reconstruct the same reasoning from the diff. on the right, the reasoning behind the code is a versioned repository artifact 15 / 25 where sce sits sce != model sce != another harness ┌──────────────────────────────────────────┐
│ Humans: intent, reason, approval │
└───────────────────┬──────────────────────┘
│
┌───────────────────▼──────────────────────┐
│ SCE: context, plans, boundaries, policy, │
│ evidence, traceability │
└───────────────────┬──────────────────────┘
│
┌───────────────────▼──────────────────────┐
│ Agent harness: Claude Code, OpenCode, Pi │
└───────────────────┬──────────────────────┘
│
┌───────────────────▼──────────────────────┐
│ Code, tests, CI, review, production │
└──────────────────────────────────────────┘
it makes human steering persistent, enforceable and portable between agents 16 / 25 the sce loop * 1. define the change (success criteria)
↓
2. draft implementation plan (task stack)
↓
* 3. review and refine the plan
↓
┌─ for each task in the plan ──────────────────────┐
│ │
│ * 4. agent session (plan one task) │
│ ↓ │
│ 5. agent session (implement that task) │
│ ↓ │
│ * 6. review and request changes │
│ │
└──────────────────────────────────────────────────┘
↓
7. run final tests and cleanup
↓
* 8. review final change and test results
↺
outer loop: review, adjust, plan a new change
* = human in the loop
17 / 25 planning as an executable contract an sce plan records: • change summary • success criteria • constraints and non-goals • atomic tasks • boundaries • verification steps • open questions the ambiguity is resolved before the plan is written: what is in scope, what
counts as success, which choices need approval, what is being assumed. 18 / 25 atomic tasks are our version of checkpoints T01 → implement → verify → human review
T02 → implement → verify → human review
T03 → implement → verify → human review
...
Final validation
each task: one goal, explicit boundaries, a clear completion condition, one coherent commit. do not wait for a 2000 loc 19 / 25 scope becomes a workflow boundary the common failure: the agent notices an adjacent problem and "helpfully" fixes it. new dependency, restructured module, changed schema, expanded refactor. so before implementation the agent restates the goal, what is in and out of scope, the completion checks, the expected files, and the risks -> then stops for approval. during execution: exactly one task by default, minimal in-scope changes, targeted checks, recorded evidence, and stop again if out-of-scope work becomes necessary. 20 / 25 deterministic backpressure instructions alone are weak. sce policy can block, before the agent acts: • direct git commit
• package-manager drift (npm in a bun repo)
• forbidden command patterns 21 / 25 what sce does not solve sce does not:
• make the model fundamentally smarter • guarantee good architecture • replace tests or ci • replace code review • provide a perfect maintainability metric • make lights-off development safe by itself it does not remove the need for steering. 22 / 25 the verification stack layer │ question ─────────────────────────┼──────────────────────────────────────── types and compiler │ is the program structurally valid? tests │ does known behaviour work? lint and static analysis │ does it violate detectable rules? ci and policy │ did it follow the required process? sce plan │ did we build the agreed change? sce context │ does it respect the system's decisions? human review │ is this a good design and trade-off? agent trace │ how and why was the change produced? several imperfect layers, each catching a different class of failure 23 / 25 the core thesis ▍ sce is the control and memory layer for ai-assisted software development
it preserves: • intent before code
• boundaries during execution
• evidence after implementation
• reasoning across time
an agent is optimized for the current context window. the human team is responsible for the system's lifecycle. 24 / 25 turn the lights on. the harness is necessary. the model is necessary. tests are necessary. none of them is sufficient. the future is not humans writing all the software. the future is also not a queue of agents generating changes no human understands. 25 / 25