Engineering demonstration

Autonomous AI repair system for open-source projects.

Agent Omega finds, localizes, fixes, and tests bugs in open-source projects. Its patches have been merged by maintainers of projects including Caddy, rclone, go-ethereum, and k3s. In one documented case, the complete repair-to-merge workflow ran without human intervention.

Explore the demo ↓ Merged PRs ↓

Under the hood

Full technical details

Pipeline architecture

Localize — a read-only model session names the faulty files from the issue text alone (88–93% top-1 on 37 ground-truth issues; keyword baseline 11%).
Patch — a bounded session (file tools, no shell) edits the checkout; only the diff is kept.
Regression test — a fix-blind session writes a test that must FAIL on base and PASS on patch, twice, stably.
Judge ×2 — independent review passes; the second reads neutralized issue text so steering produces a divergence.
16 release gates — budgets, capacity, policy, scope, lints, breaker; a gate that errors fails closed.

Correct fix vs wrong fix, side by side

The model's fix → shipped

loading…

A plausible wrong fix → blocked

loading…

Full pipeline log (unabridged)

loading…

Verification suite

loading…

Real-world reproduction: go-ethereum #34692

The seeded demo bug is synthetic by design. The same localization session, pointed at go-ethereum before the maintainers fixed issue #34692, named node/rpcstack.go top-1 in 9.6 s — matching the merged fix. One benchmark data point; localization only; live runs vary.

loading…

Numbers & limitations

Localization: 88–93% top-1 (100% top-3) on 37 issues with merged human fixes.
Fixes: validated against merged human fixes pre-launch; 4/5 decisive runs passed the maintainers' own tests.
Honest negatives: a Rust expansion closed NO-PROMOTE on pre-registered bars, twice; one PR withdrawn as a design finding.
The seeded demo bug is easier than production work; recorded output is recorded; live sessions vary.
A merged PR proves a change passed one repository's review — nothing more universal.
Reproduction tested on macOS/arm64 (go 1.25, python 3.14); Linux expected, unverified.

Security model & failure injection

Reasoning does not confer authority. Model and external text are untrusted inputs to a deterministic control plane. Six injected failures, six blocks (the demo's third tab, or make inject):

Plausible-but-wrong patch → tests catch it; tests_pass refuses.
Prompt-injection in the issue text → injection_lint refuses.
Diff touching CI config → content_lint refuses.
Oversized 40-file diff → size_caps refuses.
Tripped circuit breaker → breaker_armed refuses.
Missing state file → the gate errors and fails closed.

This site is static files only — no backend, no analytics, no cookies; nothing here can execute code or spend a model budget.

Reproduce it & raw artifacts

git clone https://github.com/SillyZir/agent-omega-lab cd agent-omega-lab make demo # full pipeline, ~1 min, no API key needed make inject # six blocked-failure scenarios make verify # machine-checkable assertions

MANIFEST.json proves by sha256 that the modules doing the work are byte-identical to the published production code; tools/record.py regenerates every recording under your own account. Artifacts: the model's patch · generated regression test · the wrong patch · judge verdicts (fix) · judge verdicts (wrong patch) · regression-test record · the seeded issue · recording transcript

Live Demonstration recorded session, deterministic replay — reproduce it: make demo

Tests FAIL on buggy code

loading…

Tests PASS on patch

loading…

✓ All 16 gates passed — SHIP PERMITTED

Inspect all 16 release gates

loading…

A seeded Go bug through the real pipeline modules. Model stages replay a captured 2026-10-21 session; tests and gates execute deterministically. Nothing runs on this site.

Production record

Merged pull requests

Every link is the public record on GitHub.

Repository	PR	Change	Status	Merged
caddyserver/caddy	#7898	Register nested named-route invokes transitively	Merged	2026-07-30
caddyserver/caddy	#7905	Flush headers immediately for SSE responses	Merged	2026-07-31
caddyserver/caddy	#7904	Isolate auth-provider responses	Merged	2026-08-01
rclone/rclone	#9639	serve nfs: fix EIO creating symlinks	Merged	2026-08-01
caddyserver/caddy	#7916	Isolate health-check state per check config	Merged	2026-08-12
caddyserver/caddy	#7920	Shield hostnames from wildcard client auth	Merged	2026-08-12
ethereum/go-ethereum	#35433	Synthesize finalized markers only with explicit sync target	Merged	2026-08-12
rclone/rclone	#9726	drive: pool resumable-upload chunk buffers	Merged	2026-08-30
rclone/rclone	#9847	onedrive: manual drive-ID fallback	Merged	2026-09-01
k3s-io/k3s	#14552	Restore Flatcar detection in install script	Merged	2026-09-03
esengine/DeepSeek-Reasonix	#9707	Merge live proxy-observed MCP tools into catalog	Merged	2026-09-25

Real upstream work Production repositories, real maintainer review — not a benchmark sandbox.

Fail-closed security Model output is a proposal; 16 deterministic gates hold ship authority. UNKNOWN is never PASS.

Fully reproducible Open lab, sha-verified production modules, one-command demo on your machine.

Honest measurement Pre-registered bars, published NO-GO verdicts, limitations stated plainly.

Judgment calls

Three design decisions

Fail closed on uncertainty

DecisionUnknown or incomplete results block shipment.

WhyA retry can hide a real failure, create duplicate actions, or turn an ambiguous outcome into a false success.

Trade-offSome recoverable jobs require manual investigation. Safety takes priority over throughput.

Give the second judge neutralized issue text

DecisionRemove attacker-controlled framing before the second review.

WhyIssue descriptions are untrusted input. The second judge evaluates the fix without being steered by instructions embedded in the report.

Trade-offNeutralization is deliberately greedy and can strip harmless context; if the two verdicts then diverge, the candidate goes to human review instead of shipping. A lost candidate is cheaper than a steered patch.

Keep the public demo recorded

DecisionReplay verified evidence rather than execute live model sessions on the public website.

WhyVisitors inspect the repair workflow without exposed credentials, untrusted code execution, or model charges.

Trade-offVisitors cannot submit new bugs to the public site. Real execution remains a controlled, local capability.

Long-form rationale: autonomy & authority in the showcase documentation.