- Home
- Skills
- Testing & Debugging
- ai output verification
Works with the AI Agents you already use
ai output verification
Audit AI-generated claims, citations, and calculations against primary sources using ISTQB-aligned QA procedures.
$6
ai output verification
Example session with this skill installed
Verify this summary of the quarterly report against the attached PDF. Check all figures for the Q3 revenue growth and ensure the citation for the market share increase is valid.
- Read your context and instructions
- Compiled the ai output verification
I have completed the verification of the quarterly report summary against the provided PDF source. I identified two numeric claims and one market share citation for audit.
AI Output Verification Report
Verdict: Revise (Critical arithmetic error in revenue calculation)
| Claim/field | Type | Evidence/source location | Support | Issue/severity | Required correction |
|---|---|---|---|---|---|
| Q3 Revenue: $4.2M | Numeric | Page 4, Section 2.1 | Contradicted | High | Source states $4.02M; summary rounded incorrectly. |
| Growth: 12% YoY | Causal | Page 5, Table 1.a | Supported | None | N/A |
| Market share: 18% | Attributed | Page 12, Paragraph 3 | Unverified | Medium | Citation points to Page 12, but text refers to Q2 data, not current Q3. |
| Data Schema | Format | N/A | Supported | None | N/A |
Arithmetic Check:
- Summary Input: $4.2M
- Source Calculation: $4,020,000 (Sum of regions A, B, and C)
- Variance: +$180,000 (Overstatement)
Limitations:
Verification was restricted to the provided PDF. No external market data was used to validate the 18% market share claim beyond the internal report context.
Next steps
- Update the summary text to reflect the $4.02M revenue figure.
- Verify if the Q2 market share data was intended or if a newer source for Q3 exists.
- Re-run the verification once the corrections are applied.
Connects securely to your tools. The creator never sees your data.
About this skill
AI-generated outputs often suffer from hallucinations, incorrect citations, and faulty logic that can break downstream workflows or violate safety protocols. Relying on model-generated summaries, code, or data extractions without a rigorous verification layer creates significant technical and compliance risks.
What it does
- Claim classification categorizes every factual, numeric, and causal claim as supported, contradicted, or unverified.
- Citation validation checks if references exist, resolve to primary source passages, and actually support the accompanying statement.
- Quantitative verification recomputes calculations, units, and date ranges independently from the model's disclosed logic.
- Schema enforcement validates structured data outputs and extraction fields against the source, identifying OCR or formatting errors.
- Risk adjudication applies task-specific rubrics to decide if an output is safe to accept, needs revision, or requires human escalation.
How it works
- Source analysis identifies all consequential claims within the AI output including legal, medical, or safety statements.
- Evidence mapping compares claims against authoritative primary sources and logs specific evidence locations.
- Independent calculation runs arithmetic and logic checks on all numeric data without assuming the model's formulas are correct.
- Final disposition generates a verification table with a formal verdict of Accept, Revise, Escalate, or Reject based on approved thresholds.
Frameworks & tools
This skill adheres to ISTQB CT-AI and CTFL standards, utilizing NIST AI Risk Management Framework principles. It integrates with any LLM workflow where structured verification against primary sources is required.
Why this beats prompting it yourself
Standard prompting often results in the model "marking its own homework" with the same biases that caused the error. This skill enforces a structured classification and independent re-calculation procedure that treats model output as unverified until primary source evidence is mapped to each specific claim.
Use cases
- Verifying RAG pipeline summaries against the original retrieved documents.
- Auditing AI-generated financial or medical reports for numeric accuracy and valid citations.
- Checking automated data extractions for OCR errors and schema compliance.
- Reviewing high-stakes recommendation outputs before human-in-the-loop approval.
Known limitations
Does not certify model performance or verify subjective claims without user-defined rubrics. Requires a writable destination folder for artifacts and access to primary source materials for grounding.
Requires primary source materials. Does not certify models or infer correctness from fluency. User must provide rubrics for subjective judgments. Outputs must stay in approved folders.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 2 days ago