More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Skill Bench: Catch Your Claude Skill's Bugs Before Buyers Do

    1

    Find out whether your Claude skill really beats plain Claude: blind with/without runs, code checks, fix briefs and a strict no-regression bar.

    $14.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Skill Bench: Catch Your Claude Skill's Bugs Before Buyers Do

    Skill Bench: Catch Your Claude Skill's Bugs Before Buyers Do

    Example session with this skill installed

    Bench my bill-splitter skill before I sell it. Three cases: an even split with tip, the same job in messy wording, and a trap where the tip % is missing. 2 runs with the skill, 1 without, graded blind.

    • Read your context and instructions
    • Compiled the bench

    Skill Bench round 1: bill-splitter (invented demo skill, synthetic cases)

    9 runs: 2 with the skill + 1 without, on 3 cases. Code checks first, then one blind grader per packet (no arm labels).

    With skillWithout skill
    Lines passed17 of 2411 of 12

    Verdict: the skill is WORSE than plain Claude on the core job.

    Finding D-1 - skill defect

    Every with-skill share is rounded down to a whole unit: 53 instead of 53.07, so the group pays 212 of 212.29. Seen in

    4 of 4 with-skill runs on the two split cases,

    0 of 2 without.

    Cause (quoted from SKILL.md step 4): "Round each share down to the nearest whole unit of the currency so nobody has to deal with coins."

    Blind grader, quoting the answer: "4 x EUR 53 = EUR 212, which is EUR 0.29 short of the grand total". Grade:

    fail.

    Test-set issue T-1

    A code check failed an answer that only mentioned a service charge. The check is dropped and the grader judges that line.

    Fix brief

    Round each share to the cent and say who pays the leftover cent.

    • Acceptance: 53.07 and 37.92 pass in 2 of 2 runs.
    • Guard: the missing-tip trap must still ask for the tip %.

    Round 2 (fixed skill)

    • 24 of 24 lines pass.
    • bench.py regress: PASS, 0 regressions, 4 lines improved.
    • Grader spot-check: 4 of 4 grades re-read against the answers agree.
    quoted from SKILL.md

    Connects securely to your tools. The creator never sees your data.

    About this skill

    A skill can pass one quick try and still fail on a repeat run, invent numbers, or make claims about itself that are not true. Skill Bench is a test method for catching that before your buyers do.

    You point it at a skill. It helps you design a test set with real spread: different inputs, messy wording, traps where the skill's own rules must hold, and planted ground truth so answers are checked, not guessed. Nothing runs until you approve the set.

    Every case runs with the skill and without it, several times. bench.py runs the code checks first (counts, lengths, numbers against a computed true value, forbidden terms, files produced). Then each answer goes to a grader in a shuffled, unlabelled packet, so the grader never knows which arm wrote it. Three grades: pass, pass with note, fail, each with a quoted reason.

    Failures are classified as skill defect, test-set issue, harness artifact or grey area, and become a fix brief with the exact old and new wording, acceptance lines and a guard line for the fix's own risk. After you change the skill, the re-bench passes only if every acceptance line holds and nothing that passed before now fails.

    It builds on Anthropic's skill-creator (its eval format, aggregator and review page) and adds the test design, blind grading, repeatability check and regression bar. It does not build skills or tune trigger descriptions; use skill-creator for that.

    Compatibility

    Best in Claude Code, or any Claude environment with sub-agents and Python 3.9+ (standard library only, no installs). Needs Anthropic's skill-creator skill for the benchmark summary and review page. In a plain claude.ai chat with no sub-agents it runs a labelled smoke test only, because one session cannot fairly write both arms. An optional API-batch route for large sets never spends without your go.

    Permissions

    • Read files: the skill you bench and your test set
    • Write files: bench.py writes only to the bench workspace you point it at
    • Delete files: only bench.py blind --force, which empties and rebuilds the packet folder you name
    • Run Python: scripts/bench.py on your own files; a python check in your test set imports and runs a check module you write
    • Sub-agents: one per run and one per grading packet
    • Network: none, except the optional bench.py fresh --links, which checks that links in your own test set still load

    Known limitations

    • It is slower than skill-creator alone. On our own bench (v1.1, 6 synthetic cases) skill-creator took 37% less time and 7% fewer tokens per run.
    • It does not build skills or tune trigger descriptions; it pairs with skill-creator.
    • A bench only measures the cases you give it. In our own bench, two trap cases did not separate the arms: plain Claude passed them too.
    • Blinding is partial: answers written with the skill can name bench.py, which hints at the arm.
    • The grader is a model. Every grade quotes its evidence, and the method asks you to spot-check a few.
    • Sub-agents can write outside their run folder (3 of 54 runs in our bench). Run it in a dedicated, empty folder.
    • The number check reads a value written after its label in a sentence; values inside a table row go to the grader instead.
    • Measured on one model family.

    Claude is a trademark of Anthropic, PBC; this skill is not affiliated with or endorsed by Anthropic.

    Requires skill-creator for full functionality. API cost estimates require manual verification if older than 30 days. Single-session runs lack a baseline comparison.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 1 month ago

    Requires

    Python

    Permissions

    Terminal / Shell
    Network Access
    Listed1 month ago
    Updated5 days ago

    What's inside

    Frequently Asked Questions

    Find out whether your Claude skill really beats plain Claude: blind with/without runs, code checks, fix briefs and a strict no-regression bar.

    This skill uses the open SKILL.md standard. Use it through an MCP-compatible agent connected to Agensi, and check the compatibility information on this page for confirmed agents and required capabilities.

    Connect your AI agent to Agensi through MCP, then ask it to use Skill Bench: Catch Your Claude Skill's Bugs Before Buyers Do on your task. Your agent loads the latest approved version each time. Paid skills require a purchase or active subscription. Skill files are not available for download; your licence does not allow you to save, copy or export any part of a skill.

    New in Testing & Debugging