- Home
- Skills
- Research & Analysis
- Experiment Plan: Claim-Driven Validation Roadmaps
More screenshots
Works with the AI Agents you already use
Experiment Plan: Claim-Driven Validation Roadmaps
Turn a research proposal into a claim-driven experiment roadmap: frozen claims, five storyline blocks, risk-ordered runs with decision gates, and an honest compute budget.
$5
Experiment Plan: Claim-Driven Validation Roadmaps
Example session with this skill installed
Turn my proposal into an experiment plan. The method is a retrieval chunking strategy that splits documents at predicted topic boundaries instead of fixed token windows; I claim it improves answer accuracy on long-document QA. I have about 300 GPU-hours, two months, and one baseline paper whose numbers I need to beat.
- Read your context and instructions
- Compiled the experiment plan
CLAIM SHEET (frozen). Primary claim: boundary-predicted chunking, not retrieval tuning, produces the accuracy gain on long-document QA. Anti-claims: (a) the gain is just smaller effective chunks; (b) the gain appears only on documents similar to the boundary predictor's training data. Minimum convincing evidence: anchor win over the strongest published baseline on held-out documents, a chunk-size-matched control answering anti-claim (a), and an out-of-domain split answering (b).
BLOCKS. (1) Anchor — proposed chunking vs. the baseline paper's fixed-window system vs. fixed-window at matched average chunk size, on the primary QA benchmark, held-out split; the matched-size arm doubles as the anti-claim (a) control. Main paper, MUST-RUN. Success: beats both arms by more than the seed interval. Failure reading: if it only beats the unmatched baseline, the claim shrinks to a chunk-size effect and the paper is repositioned before more compute burns. (2) Novelty isolation — boundary predictor replaced with random boundaries at identical chunk sizes. Main paper, MUST-RUN. (3) Failure analysis — error breakdown on the longest decile of documents and on the out-of-domain split. Appendix, answers anti-claim (b).
ORDER AND GATES. Sanity: pipeline + metric check on 50 documents (4 GPU-h). Baselines reproduced (gate: within 1 point of published numbers, else stop and resolve the setup before any method run). Anchor on a 20% slice (gate: no advantage over the matched-size arm = stop, revisit the claim). Then full anchor, isolation, failure analysis. Budget: 285 of 300 GPU-hours allocated; bottleneck is evaluation throughput. Cut order if halved: failure analysis shrinks to the out-of-domain split only; anchor and isolation are untouchable.
Connects securely to your tools. The creator never sees your data.
About this skill
Most experiment plans fail the same way: they are benchmark wishlists. Teams run everything available, fill tables, and discover at writing time that the paper's actual claims have no decisive evidence behind them — while the runs that ate the budget defended nothing. This skill plans the other way around. It freezes the claims first: one or two primary claims stated at mechanism level, the anti-claims a skeptical reviewer will raise, and the minimum convincing evidence for each. It then designs a compact storyline of at most five blocks — anchor result, novelty isolation, simplicity defense, modern-component justification, failure analysis — and assigns each to the main paper, the appendix, or the cut list. Every kept block is fully specified: claim tested, data, compared systems, metrics, setup, seeds, a written success criterion, and a written failure interpretation agreed before the result arrives, which is what keeps negative results from being buried or explained away. Runs are ordered by risk — sanity, baseline reproduction, anchor, decisive ablations, polish — with a stop/go decision gate at every milestone, so a failed core claim stops the spend instead of being decorated. The budget is planned from real constraints with an explicit cut order, and a run tracker keeps failed runs visible, because deleted failures are how phantom results happen. A plan quality rubric (claim coverage, specification, baseline strength, gate discipline, budget honesty) gates the plan before the first run starts.
- It plans experiments; it does not run them, and it never supplies or estimates results — a plan that predicts its own numbers has stopped being a plan.
- The five-block storyline is sized for a single paper's core story; a multi-paper program needs one plan per paper, not a bigger plan.
- Quality depends on the honesty of the constraints supplied: an invented budget produces a confident, useless roadmap.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 24 days ago
Permissions
No special permissions declared or detected