More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    🧠 Retrieval Trace Auditor

    1

    Audit real RAG evidence traces for missing expected sources, impossible citations, unsupported claims, stale evidence, unused context, and weak routing.

    $15

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    🧠 Retrieval Trace Auditor

    🧠 Retrieval Trace Auditor

    Example session with this skill installed

    {"case_id":"policy-17","query_class":"temporal","route":"vector_only","retrieved_evidence":[{"id":"policy-2024","superseded_by":"policy-2026"}],"expected_evidence_ids":["policy-2026"],"cited_evidence_ids":["citation-ghost"],"claims":[{"claim_id":"claim-1","text":"The 2026 policy replaced the rule.","supporting_evidence_ids":["support-ghost"]}]}

    • Read your context and instructions
    • Compiled the retrieval trace auditor

    Release gate: FAIL
    Critical: citation-ghost was cited but never retrieved.
    Critical: claim-1 declares support-ghost, which was never retrieved.
    High: policy-2026 was expected but absent; temporal query used vector_only.
    Boundary: semantic entailment and answer correctness remain unverified.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Retrieval Trace Auditor inspects the evidence path behind a RAG or knowledge-agent answer. Supply structured JSONL cases with stable retrieved evidence IDs, expected evidence when known, citations, answer claims, declared support IDs, query class, and route. The included deterministic local helper detects expected evidence that was not retrieved, citations or claim support that never appeared in context, claims with no declared support, stale selected evidence, unused retrieved context, and temporal or relational questions routed through vector-only retrieval. It returns severity-ranked findings, per-case dispositions, a release gate, and a review memo. It deliberately does not claim to judge semantic entailment or factual correctness.

    ID consistency does not prove that source text semantically entails a claim. Expected evidence is trustworthy only when an authoritative fixture or qualified reviewer established it. The input must expose stable evidence IDs and structured claim support; opaque answer-only logs cannot support the full audit.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    Permissions

    Terminal / Shell
    Read Files
    Write Files

    File Scopes

    retrieval-trace-input/**
    retrieval-trace-output/**
    *.jsonl
    *.csv
    *.json
    *.md

    The helper reads supplied trace JSONL and writes findings, case summaries, JSON, and Markdown locally. It does not call an LLM, vector store, observability platform, or external service.

    Listed3 months ago

    What's inside

    Frequently Asked Questions

    This skill identifies structural failures in the RAG pipeline, such as 'hallucinated' citations that weren't in the source text, unused context that increases latency, and instances where the retriever missed known-good documents. It helps you find technical grounding errors that traditional semantic evaluations often overlook.

    The Auditor requires structured JSONL input containing retrieved evidence IDs, query classes, and citation mapping. It is compatible with any RAG framework (like LangChain, LlamaIndex, or custom stacks) as long as your agent logs the retrieval and citation trace in a stable format.

    No, this skill is a deterministic diagnostic tool designed to audit the 'plumbing' of your RAG system. It checks if claims are mapped to existing IDs and if routing was appropriate, rather than judging if the answer is 'friendly' or 'correct' in a human sense.

    You receive the core auditing logic, a deterministic local helper for trace processing, and a structured output schema that provides severity-ranked findings and a final 'release gate' status for your production deployments.

    The Auditor is particularly effective for high-precision environments like legal, medical, or technical support, where verifying that an answer actually came from the provided context (and not the model's training data) is a critical compliance requirement.