Tests That Can Fail

    1

    A practical checklist for writing and reviewing falsifiable tests, contracts, benchmarks, and AI evaluations.

    $1.58

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    tests-that-can-fail

    Example session with this skill installed

    Review this assertion using tests-that-can-fail: result = sorted([3, 1, 2]); assert len(result) == len(result). The requirement is ascending order while preserving all input values.

    • Read your context and instructions
    • Compiled the tests-that-can-fail

    Example assertion review

    This is an illustrative review of the supplied snippet, not a test execution record.

    The assertion compares the length of result with itself. It passes for every result whose length can be read, including an unsorted list, an empty list, or a list containing the wrong values. It therefore provides no evidence for the requirement.

    A concrete wrong output is [3, 2, 1]. Another is [1, 2], which loses an input value. A replacement should check both the requested order and preservation of the values. For this fixed example, assert result == [1, 2, 3] checks both. For a general input, write separate checks for adjacent ordering and matching element counts, including duplicates.

    Before relying on the replacement, confirm it passes a known-correct result and fails the named wrong outputs. Add duplicate-value and empty-input examples when those are in scope.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Help your AI agent ask whether a test can fail, whether it passes on known-correct behavior, and whether it checks the relationship the specification actually describes.

    The package includes three questions for each check, five root-cause categories, a step-by-step writing procedure, a review checklist, guidance for evaluation harnesses, and a catalogue of 30 failure modes. Use it when drafting tests, reviewing model-written assertions, or examining a green suite that may not prove the intended behavior.

    Includes SKILL.md and references/failure-modes.md. Copy the tests-that-can-fail folder into ~/.claude/skills/ or ~/.codex/skills/.

    Source: https://github.com/SLYysl/tests-that-can-fail

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed today

    Permissions

    No special permissions declared or detected

    Listedtoday

    What's inside

    Frequently Asked Questions

    A practical checklist for writing and reviewing falsifiable tests, contracts, benchmarks, and AI evaluations.

    This skill uses the open SKILL.md standard. Use it through an MCP-compatible agent connected to Agensi, and check the compatibility information on this page for confirmed agents and required capabilities.

    Connect your AI agent to Agensi through MCP, then ask it to use tests-that-can-fail on your task. Your agent loads the latest approved version each time. Paid skills require a purchase or active subscription. Skill files are not available for download; your licence does not allow you to save, copy or export any part of a skill.

    New in Testing & Debugging