- Home
- Skills
- Research & Analysis
- Claim Check: Every Result Backed by Its Data
More screenshots
Works with the AI Agents you already use
Claim Check: Every Result Backed by Its Data
This skill makes your agent back every sentence about a result with its data: each number, verdict word and citation checked against the results file it cites.
$12
Claim Check: Every Result Backed by Its Data
Example session with this skill installed
Check my results section and claims list against the results files before I send them to my co-authors. Put a corrected copy of the results section in research/paper/results-checked.md so I can compare.
- Read your context and instructions
- Compiled the claim check
I checked the results section and the claims list against the results files, and saved a corrected copy of the results section to research/paper/results-checked.md. The claim checker gives no errors or warnings on the corrected copy. On your original it found 6 errors and 9 warnings.
"…test accuracy of 0.874 … against 0.860": TileMix's stored value is 0.87346, which rounds to
0.873, not 0.874. Split into two sentences, each with its own 95% interval and seed count.
2.
"significantly better calibrated, lowering ECE by 0.0049": the interval for the difference runs from −0.018 to 0.0078, which includes zero. Now: "−0.0049 (95% CI −0.018 to 0.0078, 5 seeds), not distinguishable from zero at this sample size."
3.
"improves accuracy by 2.8%": the stored value is 0.02755 as a fraction, so "2.8%" is a conversion done by hand, and it reads as a relative change. Now: "raised accuracy … by 0.028 (95% CI 0.019 to 0.036, 5 seeds)."
4.
"at no extra training cost": the interval from −0.69 to 2.35 minutes includes zero, but it doesn't show zero cost. Now the estimate and interval.
5.
"generalises … outperforms the baseline (0.791 vs 0.774)": each arm is a single run, with no interval and no stored difference. Now each run in its own sentence, with "one run each can't separate the gap from seed-to-seed variation".
6.
"This is because mixing tiles forces…": no experiment isolated the mechanism. Now "One explanation…, not tested here, is…"
Passed unchanged: "The paired difference was 0.013 (95% CI 0.010 to 0.016, 5 seeds)."
Claims list: C1 checked; C2–C4 revised (replacement sentences given); C5 open: needs RegionalCover runs on several seeds per arm with the paired difference stored, and even then supports "on RegionalCover", not "generalises to larger regional datasets".
Connects securely to your tools. The creator never sees your data.
About this skill
What it does
A results section is where reviewers hold your sentences against your numbers. This skill makes your agent back every sentence about a result with its data: each number, verdict word and citation checked against the results file it cites. Point it at a results paragraph, a caption, an abstract or a claims list, and it checks each claim against its evidence (the estimate, its interval, its n) and rewrites only what fails:
- Numbers match the data. Every number in a cited sentence must sit in the cited result within rounding. A mis-rounded estimate, a value copied from the wrong entry, a dropped minus sign or a fraction turned into a percent by hand is caught and fixed.
- No verdict without the evidence for it. "Improves", "outperforms" and "significantly better" need a stored difference whose interval excludes zero. "No effect", "matches" and "no extra cost" need an equivalence test, not a wide interval. The skill writes the estimate and interval instead, or "not distinguishable from zero at this sample size".
- One fact per citation. Two arms, two conditions or a raw value and its difference each get their own sentence and their own citation.
- Scope stays where it was measured. A single-seed result is reported as one run; "generalises" needs evidence at every setting; a mechanism no experiment isolated becomes "one explanation, not tested here".
- Reread before returning. An 8-step procedure, ending with a fact-by-fact reread of every claim sentence against its entry, then the checker, resolving every error it reports.
The checker
check_claims.py (Python 3.10+, standard library, no network) resolves every [results/<file>.json: <entry>] citation in your prose and claims.md and reports errors and warnings: number mismatches within rounding, hand unit conversions, verdict words over an interval that includes zero, equivalence words with no equivalence test, verdicts on n = 1 or with no interval, comparison words with no stored difference, citations that cover several results, and broken citations. In the demo it found 6 errors and 9 warnings in a draft results section and 0 in the corrected copy.
How it differs from citation auditors
Citation auditors check that a reference supports a sentence. Claim Check checks your own results: it opens the JSON your sentence cites and verifies the numbers and verdict words against the stored value, interval and n.
What's included
SKILL.md: the citation syntax, the 8-step procedure, a quick-reference table and worked examplesreferences/verdict-words.md: nine families of verdict words (comparison, significance, equivalence, magnitude, mechanism, scope, trend, confirmation, ranking) and what evidence each needsreferences/hard-cases.md: phrasing for null results, overlapping intervals, single runs, n = 1, missing intervals, many comparisons, percent versus percentage pointsscripts/check_claims.py: the deterministic checker
Part of the Aurora Lab research pipeline
Works entirely on its own: with a research/ folder it reads research/results/*.json and updates research/claims.md; without one it works from the numbers you paste. Pairs well with Aurora Lab's Scientific Voice (how sentences read) and Test Readers (how a cold reader takes the paper), but needs neither.
It checks claims against the evidence they cite; it does not run experiments or compute statistics, and it never invents a missing number: a claim that needs a result you have not stored (such as a paired difference or an equivalence test) is marked open with exactly what is missing. The checker matches verdict words from a fixed list, so a paraphrase ("edges out", "a clear gain") passes it and an innocent use ("we increased the batch size") draws a warning. A number passes the checker when it equals any value stored in the cited entry within the rounding it is written at, interval bounds included, so a mis-rounded estimate that equals a bound, or a small whole number that equals an unrelated count, slips through; numbers written as words are not checked. The checker cannot judge scope, mechanism or whether the right entry was cited: the agent's reread covers those, and it is a careful review, not a guarantee. The checker reads citations in the [results/<file>.json: <entry>] form; other ways of pointing at evidence are checked by the agent only. It does not check references to the literature.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 29 days ago
Requires
Permissions
File Scopes
Reads the results files your sentences cite (research/results/*.json), your claims list (research/claims.md) and the prose you ask it to check (research/paper/*, figure captions), or numbers you paste. Writes claims.md (sentences and status) and edits paper text or captions only when you ask for edits rather than a review; a corrected copy goes where you say. The checker reads files and writes nothing. Never modifies your results files. No network access.