More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Experiment Plan: Claim-Driven Validation Roadmaps

    1

    Turn a research proposal into a claim-driven experiment roadmap: frozen claims, five storyline blocks, risk-ordered runs with decision gates, and an honest compute budget.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Experiment Plan: Claim-Driven Validation Roadmaps

    Experiment Plan: Claim-Driven Validation Roadmaps

    Example session with this skill installed

    Turn my proposal into an experiment plan. The method is a retrieval chunking strategy that splits documents at predicted topic boundaries instead of fixed token windows; I claim it improves answer accuracy on long-document QA. I have about 300 GPU-hours, two months, and one baseline paper whose numbers I need to beat.

    • Read your context and instructions
    • Compiled the experiment plan

    CLAIM SHEET (frozen). Primary claim: boundary-predicted chunking, not retrieval tuning, produces the accuracy gain on long-document QA. Anti-claims: (a) the gain is just smaller effective chunks; (b) the gain appears only on documents similar to the boundary predictor's training data. Minimum convincing evidence: anchor win over the strongest published baseline on held-out documents, a chunk-size-matched control answering anti-claim (a), and an out-of-domain split answering (b).

    BLOCKS. (1) Anchor — proposed chunking vs. the baseline paper's fixed-window system vs. fixed-window at matched average chunk size, on the primary QA benchmark, held-out split; the matched-size arm doubles as the anti-claim (a) control. Main paper, MUST-RUN. Success: beats both arms by more than the seed interval. Failure reading: if it only beats the unmatched baseline, the claim shrinks to a chunk-size effect and the paper is repositioned before more compute burns. (2) Novelty isolation — boundary predictor replaced with random boundaries at identical chunk sizes. Main paper, MUST-RUN. (3) Failure analysis — error breakdown on the longest decile of documents and on the out-of-domain split. Appendix, answers anti-claim (b).

    ORDER AND GATES. Sanity: pipeline + metric check on 50 documents (4 GPU-h). Baselines reproduced (gate: within 1 point of published numbers, else stop and resolve the setup before any method run). Anchor on a 20% slice (gate: no advantage over the matched-size arm = stop, revisit the claim). Then full anchor, isolation, failure analysis. Budget: 285 of 300 GPU-hours allocated; bottleneck is evaluation throughput. Cut order if halved: failure analysis shrinks to the out-of-domain split only; anchor and isolation are untouchable.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Most experiment plans fail the same way: they are benchmark wishlists. Teams run everything available, fill tables, and discover at writing time that the paper's actual claims have no decisive evidence behind them — while the runs that ate the budget defended nothing. This skill plans the other way around. It freezes the claims first: one or two primary claims stated at mechanism level, the anti-claims a skeptical reviewer will raise, and the minimum convincing evidence for each. It then designs a compact storyline of at most five blocks — anchor result, novelty isolation, simplicity defense, modern-component justification, failure analysis — and assigns each to the main paper, the appendix, or the cut list. Every kept block is fully specified: claim tested, data, compared systems, metrics, setup, seeds, a written success criterion, and a written failure interpretation agreed before the result arrives, which is what keeps negative results from being buried or explained away. Runs are ordered by risk — sanity, baseline reproduction, anchor, decisive ablations, polish — with a stop/go decision gate at every milestone, so a failed core claim stops the spend instead of being decorated. The budget is planned from real constraints with an explicit cut order, and a run tracker keeps failed runs visible, because deleted failures are how phantom results happen. A plan quality rubric (claim coverage, specification, baseline strength, gate discipline, budget honesty) gates the plan before the first run starts.

    • It plans experiments; it does not run them, and it never supplies or estimates results — a plan that predicts its own numbers has stopped being a plan.
    • The five-block storyline is sized for a single paper's core story; a multi-paper program needs one plan per paper, not a bigger plan.
    • Quality depends on the honesty of the constraints supplied: an invented budget produces a confident, useless roadmap.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    Permissions

    No special permissions declared or detected

    Listed24 days ago
    Updated21 days ago

    What's inside

    Frequently Asked Questions

    No. Each run must name the claim it defends and what outcome would change a reviewer's belief. Runs that cannot do that get cut or moved to the appendix list, however interesting they are. Expect the list to shrink; that is the point.

    That is what the decision gate is for. The plan defines, in advance, what a negative anchor means and what gets decided there — reposition the claim, fix the method, or stop — instead of spending the remaining budget decorating a result that does not support the paper.

    No. Planning after the fact converts a roadmap into a justification, and the failure interpretations written in advance are what give the plan its force. For work already run, use an audit or tracking process instead; this skill is for decisions not yet spent.

    New in Research & Analysis