Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    AI Answer Grounding Check: Verify RAG Claims vs Sources

    1

    RAG systems and agents often retrieve the correct documents yet still generate hallucinations or misaligned claims.

    $9.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    AI Answer Grounding Check: Verify RAG Claims vs Sources

    AI Answer Grounding Check: Verify RAG Claims vs Sources

    Example session with this skill installed

    Verify this RAG answer. Answer: 'Plan B costs $50/mo.' Source 1: 'Plan B is $40 per month plus a $10 service fee.' Source 2: 'Plan A is $50.' Question: 'How much is Plan B?'

    • Read your context and instructions
    • Compiled the ai answer grounding

    The answer for Plan B has been verified against the provided sources. While the total amount is mathematically correct, the grounding is only partial due to the omission of the fee structure mentioned in Source 1.

    Grounding Audit

    QUESTION
    How much is Plan B?

    CLAIMS TABLE

    ClaimLabelPassage IDQuoteNote
    Plan B costs $50/moPARTLY SUPPORTEDSource 1"$40 per month plus a $10 service fee"The total matches, but the answer ignores the base price vs fee distinction.

    CITATION CHECK
    No citations were provided in the original answer. A citation to Source 1 is required for the pricing claim.

    NUMBERS & NAMES CHECK

    • Plan B: Matches Source 1.
    • $50: Calculation ($40 + $10) is correct based on Source 1.
    • Monthly frequency: Matches Source 1.

    FAILURE CAUSES
    The model performed the correct arithmetic but simplified the pricing structure. This is a "reasoning/simplification" issue rather than a retrieval failure.

    SCORE: 0 of 1 fully supported, 1 partly supported. Severity: Low.

    GROUNDED REWRITE
    Plan B costs $40 per month plus a $10 service fee, totaling $50 per month [Source 1].

    EVAL CASES

    {"question": "How much is Plan B?", "context": "Plan B is $40 per month plus a $10 service fee.", "ideal_answer": "Plan B is $40/mo with a $10 fee.", "rejected_facts": ["Plan B is $50 flat"]}
    

    Next steps

    1. Update your system prompt to require explicit mentions of service fees.
    2. Add this example to your regression test suite using the JSONL output.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    RAG systems and agents often retrieve the correct documents yet still generate hallucinations or misaligned claims. This skill provides a systematic way to verify AI-generated answers against their source passages, ensuring every statement is anchored in your data.

    What it does

    • Claim decomposition splits the answer into atomic facts, preserving specific numbers, dates, and qualifiers.
    • Evidence matching labels every claim as supported, partly supported, unsupported, or contradicted with exact quotes.
    • Citation audit verifies that existing citations actually point to the relevant passage and flags missing references.
    • Technical verification re-checks arithmetic, units, currencies, and proper noun spellings against source text.
    • Root cause analysis identifies if failures stem from retrieval gaps, model reasoning, or conflicting documents.
    • Grounded rewriting produces a corrected version of the answer containing only verified information.

    How it works

    1. Provide context by submitting the original question, the AI's answer, and the retrieved source passages.
    2. Review the verification table which maps each atomic claim to its specific supporting quote and passage ID.
    3. Analyze failure causes to determine if you need to fix your chunking strategy, retrieval logic, or system prompt.
    4. Generate eval sets by converting check results into JSONL format for regression testing.

    Frameworks & tools

    Works with any RAG architecture or agentic workflow. Optimized for verifying outputs from LLMs like Claude or GPT before they reach end users or are stored in evaluation logs.

    Why this beats prompting it yourself

    Standard prompts often miss "meaning drift" where an LLM turns a conditional statement into a certainty. This skill forces a claim-by-claim audit that prevents the model from grading its own homework with the same biases that caused the hallucination.

    Use cases

    • Audit support bots to ensure technical documentation is reflected accurately in customer replies.
    • Legal and compliance review of AI-generated summaries against source contracts or policies.
    • RAG evaluation to build high-quality golden datasets from raw production logs.
    • Pre-release testing of medical or financial agents where accuracy is non-negotiable.

    Known limitations

    Requires the user to provide the exact passages used by the system. It cannot verify claims without the source text.

    Requires the specific retrieved passages used by the AI to perform verification. Not a general-purpose fact-checker for external web data.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 16 days ago

    Permissions

    Read Files
    Listed16 days ago

    What's inside

    Frequently Asked Questions

    The question, the answer and the retrieved passages or tool results, each with an id. Without the passages it asks for them.

    No. It judges only against the passages you give it. A claim the passages don't state is marked unsupported, not false.

    Yes. It can turn a batch of checks into JSONL regression cases with expected facts and facts that must not appear.

    Yes. For each failure it names the likely cause: retrieval missed it, the passage was ignored, two passages were blended, or the gap was filled from general knowledge.

    New in LLM Ops