← All case studies
Self-Optimising Agent Harness
Ouroboros
Hackathon build · measured in CI on every push
An agent harness that reads its own execution traces, proposes a cheaper strategy, and keeps the change only when an independent verifier confirms the task still passes.
-67.75%
Arithmetic task
7,479 → 2,412 tokens, task still passing
-63.13%
Text-slug task
6,531 → 2,408 tokens, task still passing
2
Independent benchmark tasks
both start from a genuinely failing suite
The Problem
- Agent runs get more expensive over time, and nobody can tell which part of the strategy is actually wasting tokens.
- Changing an agent's strategy is risky. A silent regression costs more than the saving, so the waste tends to stay in place.
The Solution
- The agent records every model call, tool call and phase as a span, then reads its own trace to find where the cost actually went.
- It proposes a change from a closed set of strategy fields, and every proposal cites the span that justified it.
- The task is re-run from a pristine snapshot of the sandbox, so the comparison is against a clean baseline rather than a polluted one.
- An independent verifier re-runs the task's own objective grader from a clean tree and compares an artifact hash before and after. It reads exit codes and the filesystem, never the agent's own summary of what it did.
- If the verifier does not confirm the task, the change is reverted and the reason is written down, so the system remembers what did not work.
The Result
- Arithmetic task: 7,479 → 2,412 total tokens, a 67.75% reduction, with the task still passing.
- Text-slug task: 6,531 → 2,408 total tokens, a 63.13% reduction, with the task still passing.
- Both figures are produced by CI on every push rather than pasted from a local run, so they cannot quietly drift.
- Both tasks start from a genuinely failing test suite and end passing, so the reduced token count is never the claim on its own.
- The revert is the point: on the next cycle the optimiser proposes a further change, the verifier refuses to confirm the task, and the agent reverts it instead of shipping a cheaper incorrect run.
What this does not claim
- The offline demonstration replays a scripted plan, so the comparison measures strategy efficiency given a plan, not search or reasoning. The plan is identical in both runs and only the strategy differs.
- Running against a live model is supported, and that is where a reasoning claim would have to be made. It has not been made here.
- No users, no deployment and no revenue. This is a hackathon submission with a reproducible benchmark and a public CI log.
Stack
PythonNeatlogsEntireCIMCP