- Home
- Skills
- Code Quality & Review
- Oracle-First Agent Coding
Works with the AI Agents you already use
Oracle-First Agent Coding
Agent code changes that only ever make more tests pass: build and lock the grader first, keep a change only if nothing breaks.
$19
Oracle-First Agent Coding
Example session with this skill installed
Port our Python slugify() library to TypeScript and make sure it behaves exactly the same.
- Read your context and instructions
- Compiled the oracle-first agent coding
Summary
Oracle-first: the Python library is the oracle. I built the grader before writing any TypeScript.
| Step | Result |
|---|---|
| Oracle | Python slugify() on 500 inputs (unicode, emoji, separators, max_length, stopwords) |
| Grader validated | Python vs Python: 500/500 |
| Baseline (empty TS module) | 0/500 |
| Oracle locked | gate.py lock fixtures/ grader/ -o oracle.lock.json (502 files) |
Ratchet log
| Round | Change | Passing | Newly failing | Gate |
|---|---|---|---|---|
| 1 | ASCII lowercasing + separator | 0 -> 212 | 0 | KEEP |
| 2 | unicode transliteration table | 212 -> 431 | 0 | KEEP |
| 3 | max_length word-boundary cut | 431 -> 470 | 3 | REJECT (broke 3 emoji cases) |
| 3b | same, emoji-safe | 431 -> 474 | 0 | KEEP |
| 4 | stopwords + custom replacements | 474 -> 500 | 0 | KEEP |
$ python3 scripts/gate.py verify oracle.lock.json
oracle intact: 502 files match oracle.lock.json
$ python3 scripts/gate.py compare results/start.json results/final.json
passing: 0 -> 500 (+500) newly passing: 500 newly failing: 0 missing from candidate: 0
KEEP
Next steps
- Add the grader to CI so any future change must pass
gate.py compareagainst main. - Keep the fixtures locked; add new cases in
fixtures/extra/and re-lock in a separate commit.
Connects securely to your tools. The creator never sees your data.
About this skill
Outcome: your agent only keeps a change when more tests pass and none break, so the codebase never quietly gets worse.
Why this over vibe coding: the grader is built and proven first, then locked, so the agent can't move the goalposts.
Guaranteed: if it doesn't do what this page says on your setup, message me here or email support@tensorust.com and I fix it.
What it prevents
Long agent runs fail the same way: lots of plausible code, the agent's own tests pass, and nobody can say whether it works. Or the build goes red and the agent edits the tests. This skill makes "done" a number from a grader the agent cannot touch.
Where it comes from (true project)
An agent team was set up to write a SQL engine in Rust, graded by SQLite's own sqllogictest suite, and the grader was built before any engine code:
- Corpus: 17 upstream files, 11,185 records, byte-identical to upstream, SHA-256 hashes stored.
- Grader validated on the oracle: real SQLite through the same grader passed 11,183 of 11,185; the 2 misses were traced to a SQLite version change, so the grader is trusted.
- Baseline understood: the stub engine scored 30 of 11,185, all records that expect an error, which a stub that rejects everything passes for free.
- Ratchet: agents may write only the source directory; a change is kept only if more tests pass and no passing test breaks.
And in a small Claude Code experiment with contradictory tests, 2 of 6 runs edited a test to force green. A locked oracle and an outside grader make that worthless.
What you get
- The 6-step workflow: find the oracle (reference impl, upstream suite, recorded behavior, pre-refactor snapshots), validate the grader on it, record and explain the baseline, lock it, ratchet, report numbers.
- An oracle table for ports, parsers, API clones, refactors, migrations, bug fixes.
gate.py(stdlib Python):compare base candidate --require-improvement-> KEEP or REJECT with the exact tests that broke- tests that disappear count as broken, so shrinking the suite cannot fake a KEEP
lock tests/ corpus/ -o oracle.lock.jsonandverify-> catches any edited, added or deleted oracle file- auto-detects JUnit XML, JSON, cargo/pytest/go output
- Agent instructions and a done-checklist that demand grader numbers, not adjectives.
Free companion from the same author: tamperproof rules (existing tests read-only, done means the real suite passes).
Needs an oracle (reference implementation, upstream suite or recorded behavior). gate.py compares results; it does not run your tests.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 1 month ago