More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Paper Claim Audit: Numbers vs. Evidence

    1

    Reconcile a paper against its own raw result files: every number, comparison, and scope claim extracted, recomputed, and graded FAIL, WARN, or NOTE before submission.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Paper Claim Audit: Numbers vs. Evidence

    Paper Claim Audit: Numbers vs. Evidence

    Example session with this skill installed

    Audit the claims in my results section before submission. The abstract says we improve accuracy by 12% over the strongest baseline, the main table reports 84.3 ± 0.4 against the baseline's 83.9, and the text says the method is consistently best across all datasets. My raw result files are in the project folder.

    • Read your context and instructions
    • Compiled the paper claim audit

    CLAIM AUDIT — results section + abstract, auditor working from paper and raw files only. 11 checkable claims extracted from the excerpt; 3 findings.

    FAIL — C1, headline number. Claim: "improves accuracy by 12% over the strongest baseline." Evidence: no raw file supports a 12% figure. The run logs show five seeds for the new method (best 89.1, mean 84.3) and a baseline mean of 80.9 in the earliest config; against the baseline actually reported in the main table (83.9), the honest mean-versus-mean delta is 0.4 points, and the 12% traces to comparing the best seed against a superseded baseline run. Fix: replace the headline with the mean-versus-mean figure from the reported comparison, or re-run. Verdict impact: NOT READY while C1 stands.

    WARN — C4, comparison wording. Claim: "outperforms the baseline (84.3 ± 0.4 vs 83.9)." The point estimates favor the method, but the baseline's interval (±0.9 in the raw file) overlaps; the unqualified wording overstates a 0.4-point gap. Fix options recorded: qualify ("comparable, with a small mean advantage") or add seeds until the intervals separate. Decision required from the authors; silence does not count as acceptance.

    WARN — C7, scope claim. Claim: "consistently best across all datasets." Evidence: best on 4 of 6 datasets; the two losses are in Appendix C runs whose files are present and whose results the main text never mentions. Fix: qualify to "best on four of six datasets" and discuss the losses, or narrow the claim to the benchmark family where it holds.

    Verified sample: Table 1's remaining per-dataset numbers match the raw logs exactly, and the ablation delta recomputes correctly. Verdict: NOT READY (1 FAIL, 2 WARN awaiting decisions).

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Papers drift from their evidence in ways authors cannot see: the headline improvement quietly compares a best seed against a baseline mean, an ablation table survives from a configuration the methods section no longer describes, "consistently best" forgets the two datasets in the appendix where the method lost. This skill is the reconciliation protocol that catches the drift. An auditor who is not the author — a colleague, a second model, a fresh session — receives only the paper and the raw result files, and extracts every checkable claim into a ledger: every number, every comparison rewritten as a numerical relationship, every scope word with its set enumerated, every method description, every load-bearing citation. Each claim is mapped to the evidence file that should support it; a claim with no file is not unverified, it is unsupported, and that is a finding. Verification means locating the exact value in the raw file, recomputing every derived quantity (means, deltas, error bars) from raw values rather than trusting the paper's arithmetic, and checking comparisons against the error bars before any "outperforms" is accepted. Findings are graded FAIL (contradicts the evidence; blocks submission), WARN (defensible but misleading; needs a recorded accept-or-fix decision), or NOTE (minor). The verdict follows fixed rules: any FAIL outstanding means NOT READY. Ledger template, audit checklist, severity table, and a worked example of a 47-claim audit are included.

    • It verifies that claims match evidence; it does not judge whether the method is sound, the experiments well designed, or the work novel — that is peer review.
    • Verification is only as strong as the evidence supplied: result files that were themselves fabricated pass a consistency check, which is why the audit grades provenance problems (missing files, version skew) as findings rather than assuming good faith.
    • Citation spot-checks cover load-bearing citations, not the full bibliography.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    Permissions

    No special permissions declared or detected

    Listed24 days ago
    Updated21 days ago

    What's inside

    Frequently Asked Questions

    Because the failure mode is not carelessness, it is knowledge: you remember the good seed and read tables as confirming the story. The protocol therefore requires an auditor who is not the author, working only from the paper and the raw files. If you must self-audit, use a fresh session with no project context and treat the result as weaker.

    Yes. The audit distinguishes "unverified" from "unsupported": if no evidence file exists, there is nothing to verify against, and the claim rests on memory. Re-run the experiment to produce the file, or remove the claim.

    It made the paper defensible. A 4.1% improvement that survives a reviewer's own recomputation is worth more than a 12% figure that collapses the first time someone opens the result files — and reviewers do open them.

    New in Research & Analysis