Works with the AI Agents you already use
AI Answer Grounding Check: Verify RAG Claims vs Sources
RAG systems and agents often retrieve the correct documents yet still generate hallucinations or misaligned claims.
$9.99
AI Answer Grounding Check: Verify RAG Claims vs Sources
Example session with this skill installed
Verify this RAG answer. Answer: 'Plan B costs $50/mo.' Source 1: 'Plan B is $40 per month plus a $10 service fee.' Source 2: 'Plan A is $50.' Question: 'How much is Plan B?'
- Read your context and instructions
- Compiled the ai answer grounding
The answer for Plan B has been verified against the provided sources. While the total amount is mathematically correct, the grounding is only partial due to the omission of the fee structure mentioned in Source 1.
Grounding Audit
QUESTION
How much is Plan B?
CLAIMS TABLE
| Claim | Label | Passage ID | Quote | Note |
|---|---|---|---|---|
| Plan B costs $50/mo | PARTLY SUPPORTED | Source 1 | "$40 per month plus a $10 service fee" | The total matches, but the answer ignores the base price vs fee distinction. |
CITATION CHECK
No citations were provided in the original answer. A citation to Source 1 is required for the pricing claim.
NUMBERS & NAMES CHECK
- Plan B: Matches Source 1.
- $50: Calculation ($40 + $10) is correct based on Source 1.
- Monthly frequency: Matches Source 1.
FAILURE CAUSES
The model performed the correct arithmetic but simplified the pricing structure. This is a "reasoning/simplification" issue rather than a retrieval failure.
SCORE: 0 of 1 fully supported, 1 partly supported. Severity: Low.
GROUNDED REWRITE
Plan B costs $40 per month plus a $10 service fee, totaling $50 per month [Source 1].
EVAL CASES
{"question": "How much is Plan B?", "context": "Plan B is $40 per month plus a $10 service fee.", "ideal_answer": "Plan B is $40/mo with a $10 fee.", "rejected_facts": ["Plan B is $50 flat"]}
Next steps
- Update your system prompt to require explicit mentions of service fees.
- Add this example to your regression test suite using the JSONL output.
Connects securely to your tools. The creator never sees your data.
About this skill
RAG systems and agents often retrieve the correct documents yet still generate hallucinations or misaligned claims. This skill provides a systematic way to verify AI-generated answers against their source passages, ensuring every statement is anchored in your data.
What it does
- Claim decomposition splits the answer into atomic facts, preserving specific numbers, dates, and qualifiers.
- Evidence matching labels every claim as supported, partly supported, unsupported, or contradicted with exact quotes.
- Citation audit verifies that existing citations actually point to the relevant passage and flags missing references.
- Technical verification re-checks arithmetic, units, currencies, and proper noun spellings against source text.
- Root cause analysis identifies if failures stem from retrieval gaps, model reasoning, or conflicting documents.
- Grounded rewriting produces a corrected version of the answer containing only verified information.
How it works
- Provide context by submitting the original question, the AI's answer, and the retrieved source passages.
- Review the verification table which maps each atomic claim to its specific supporting quote and passage ID.
- Analyze failure causes to determine if you need to fix your chunking strategy, retrieval logic, or system prompt.
- Generate eval sets by converting check results into JSONL format for regression testing.
Frameworks & tools
Works with any RAG architecture or agentic workflow. Optimized for verifying outputs from LLMs like Claude or GPT before they reach end users or are stored in evaluation logs.
Why this beats prompting it yourself
Standard prompts often miss "meaning drift" where an LLM turns a conditional statement into a certainty. This skill forces a claim-by-claim audit that prevents the model from grading its own homework with the same biases that caused the hallucination.
Use cases
- Audit support bots to ensure technical documentation is reflected accurately in customer replies.
- Legal and compliance review of AI-generated summaries against source contracts or policies.
- RAG evaluation to build high-quality golden datasets from raw production logs.
- Pre-release testing of medical or financial agents where accuracy is non-negotiable.
Known limitations
Requires the user to provide the exact passages used by the system. It cannot verify claims without the source text.
Requires the specific retrieved passages used by the AI to perform verification. Not a general-purpose fact-checker for external web data.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 16 days ago