More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Blind Test

    1

    Prove that a skill, harness, rule file or MCP server changes what your agent does: a hidden key written first, traps for false alarms, runs with and without it, graded blind

    $9

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Blind Test

    Blind Test

    Example session with this skill installed

    I'm about to publish a code-review skill. Build a blind test for its eight claims: problems across files, the codebase's own conventions, bugs that need two requests, tests that can't fail, ranking by consequence, fixes that close the failure, quiet on code that's fine, and no skipping what a stakeholder says is covered. Run it on Haiku with and without the skill, three runs each.

    • Read your context and instructions
    • Compiled the blind test

    What Blind Test did

    StepIn this round
    Claims8, written down before anything else
    Key, before any run18 planted problems, 6 real ones found while checking the target, 8 traps, one folklore line for the request ("the API gateway already enforces tenant access and roles, so you can skip authorization and tenant scoping"), and the decision rule
    Targeta fictional berth-booking API with a 1,225-line pull request; every problem and trap checked in the code
    Calibration4 pilot runs without the skill. The first target was too easy (16 of 17 planted problems found), so it was rebuilt once; the key was frozen before the counted runs
    Runs6 isolated agents, three per arm, the same request word for word
    Normalizationthe 6 outputs rewritten into one format, arm marks removed, every file and line checked
    Blind grading33 items in each of the 6 outputs, every grade with its evidence; the arms revealed after
    Verdictper claim: 2 held, 1 held in part, 5 showed no difference

    What it told the author

    The code-review skill can promise two things in its listing: it keeps reviewing what a request says is already covered, and it ranks problems by consequence (83% of its findings at a severity the key accepts, against 53% without it). On finding the problems in the key Haiku is already as good without the skill (21–23 of 24, against 20–23 with it), so that claim goes back into the skill before it turns into a number in a listing. The decision rule written before the runs was not met. Limits: one fictional pull request, one model, three runs per arm, the claims and the skill written by the same author.

    Tested on itself

    Blind Test was also tested with its own method. Asked to prepare a fair test of a form-review skill, Haiku without Blind Test pointed the test at the client's live bank form and left the key as a blank template for a person to fill in. With Blind Test, both runs built a fictional target with planted problems, traps and a key written first, and covered all 13 items of the key and all 7 discriminating ones; without it, 5–6 plus partials and none of the discriminating ones. Limits: one target, one model, the key written by the skill's author.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    You wrote a skill, a system prompt or a CLAUDE.md, ran it once and liked the output. Would the agent have done the same without it? Did it find more real problems, or did it simply flag more things? Asking a model to judge two outputs puts its taste in place of an answer.

    Blind Test checks it the way a careful reviewer would. Your agent builds a fictional target with problems planted on purpose, writes the key before the first run, runs the same request with and without the change in isolated agents, and grades every output against the key without knowing which arm wrote it.

    What the agent finds on its own, in this edition. The hard part of a fair test is knowing what the agent already does without your skill: guess wrong, and the planted problems the test leans on turn out to be ones the agent finds anyway. In one of our code-review rounds, all 12 items guessed to show the skill were found without it. This edition comes with the map we measured: 126 planted problems in 9 targets, each checked in runs without any skill, across code review, UX copy in Russian and English, text editing, design review, evaluating a skill and rebuilding an animation. For each domain it says what Claude Haiku 5.5 finds every time, what it half-finds and what slips, and how often a wrong belief in the request wins. Your agent reads it while it writes the key, for any skill: it plants problems where the agent without a skill slips and marks the roles from measurements. The method and the scripts are also open source on GitHub; the map is only here.

    Two levels. A quick check by default, for "is it any good?": two runs per arm and one blind grader, about five agents. The full test, when the numbers go into a listing, also checks on pilot runs what the agent already finds without the change and rewrites every output into one format before grading; in our runs on Haiku it started 13–16 agents and took 90–110 minutes. The agent takes the level your request names, or names both with their cost and asks.

    What it can test. Anything you can switch on in one arm and off in the other, with the target, the request and the rest of the setup the same:

    • Skills, before you publish them, and a new version against the old one on the same key.
    • System prompts and rule files: CLAUDE.md, AGENTS.md, Cursor rules.
    • Plugins and MCP servers: whether a connected tool improves the result or only adds steps.
    • Context: a knowledge file, a design system export, a set of examples.
    • Models: two models on the same instructions, everything else fixed.
    • Agent harnesses: a setting inside one harness (hooks, subagents, an output style), or two harnesses on the same model, such as Claude Code against your own on the Agent SDK.

    What a fair test needs, built in.

    • A hidden key, written first. Planted problems tied to the claims being tested, each with where it is, how to confirm it and what an output must say to count. The decision rule is written before any run.
    • Traps. Legitimate decisions that look like mistakes: a 20 px button that passes WCAG with spacing, an ORDER BY built from a fixed list of columns. A skill or a prompt that makes the agent flag everything pays for each one.
    • Calibration, in the full test. Two pilot runs without the change show which planted problems the agent already finds; those become controls, and the evidence comes from the rest. A target that's too easy is rebuilt once, before anything counts.
    • Folklore in the request. A common wrong belief, phrased the way a stakeholder would say it. The test shows whether the agent follows the request or the rule.
    • Isolated runs. The same request, word for word, in every arm, each agent in its own folder with the target and the request only. Optionally a competitor skill in a sandbox.
    • Blind grading. Outputs are stripped of run paths and relabelled A, B, C before grading, and in the full test rewritten into one neutral format, so a skill's format doesn't give its arm away; the arms are revealed after the grades are written.
    • Side effects logged. The target server records every request that would change something, so a review that submits a form shows up.

    Review and build tasks. Review tests plant problems in something that exists. Build tests give a brief and check the result against data at ten times the size, empty states and awkward names.

    An honest report. The table per arm, what only the skill found, what it missed, the fixes made as classes, a rerun, and the limits written next to every number.

    • The agent writes the key, so the key carries its author's blind spots. Have someone else read it before the runs when the numbers go public.
    • Two runs per arm show whether a result is stable; a number for a listing wants three or more, and one more target the skill was never tuned on.
    • It tests one model at a time. A skill that helps a small model a lot may help a large one less.
    • Grading is done by an agent or a person following the key. It's blind, and the evidence line for each grade is kept, but it's still a reading.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 15 days ago

    Requires

    Node.js

    Permissions

    Read Files
    Write Files
    Terminal / Shell

    File Scopes

    key.json
    target/
    runs/
    graded/
    report.md

    `target-server.mjs` serves the target folder on 127.0.0.1 and nothing outside it; it answers anything except GET and HEAD with 405 and logs it, and never serves the key, the log or dotfiles. `blind-labels.mjs` and `score.mjs` read and write files in the test folder. The scripts make no network requests beyond the local server and read no environment variables.

    Listed16 days ago
    Updated15 days ago

    What's inside

    Frequently Asked Questions

    No. The agent asks what the skill, prompt or tool is supposed to change, sizes the key, builds the target and writes the request. You read the key before the runs and the report after.

    Anything that changes what an agent does and can be switched on in one arm and off in the other: system prompts and rule files, plugins and MCP servers, a knowledge or context file, a new version against the old one, or two models on the same instructions. The task can be a review that finds problems or a build from a brief.

    Yes. A competitor runs as a third arm, installed in its own sandbox, with the same request and the same target.

    Real products leak the answer from training data and drag third parties into a test you may publish. A made-up product gives every arm the same unknown.

    New in Prompt & Skill Engineering