Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    SledTrace RAG Check

    1

    You get a focused investigation plan instead of an unsupported verdict.

    $4.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    SledTrace RAG Check

    SledTrace RAG Check

    Example session with this skill installed

    Run SledTrace RAG Check on this synthetic example. Identify numeric evidence conflicts, show exact quotations, and explain what needs human verification.

    {"question":"How many days do customers have to return an item?","answer":"Customers can return an item within 45 days.","chunks":[{"id":"policy-a","source":"policy-a.md","text":"Customers can return an item within 30 days."},{"id":"policy-b","source":"policy-b.md","text":"Customers can return an item within 14 days."}]}
    
    • Read your context and instructions
    • Compiled the sledtrace rag check

    Synthetic built-in demonstration.

    SledTrace RAG Check

    Checker version: 0.1.0. Findings: 2.

    Input coverage

    • chunks_supplied: 2
    • usable_chunks: 2
    • answer_present: True
    • answer_nonempty: True
    • scores_present: 0
    • eligible_context_quantities: 2

    Findings

    1. conflicting_numeric_evidence

    Context chunks contain different quantities in similar local wording.

    Evidence field: chunks[0].text
    Chunk ID: policy-a
    Source label: policy-a.md

    Customers can return an item within 30 days

    Evidence field: chunks[1].text
    Chunk ID: policy-b
    Source label: policy-b.md

    Customers can return an item within 14 days

    Details: {"shared_terms": ["customers", "item", "return"], "unit": "day", "values": [["30", "30"], ["14", "14"]]}
    Next check: Compare dates, scope and conditions; establish source authority before choosing a value.

    2. answer_numeric_mismatch

    An answer quantity differs from comparable context quantities; no exact match was found.

    Evidence field: answer

    Customers can return an item within 45 days

    Evidence field: chunks[0].text
    Chunk ID: policy-a
    Source label: policy-a.md

    Customers can return an item within 30 days

    Details: {"answer_value": ["45", "45"], "example_context_value": ["30", "30"], "unit": "day"}
    Next check: Check the exact question, ranges, conditions and any calculation before treating this as an error.

    Check coverage

    • no_retrieved_chunks: ran
    • duplicate_chunks: ran
    • low_retrieval_score: skipped — No explicit, calibrated higher-is-better score threshold.
    • weak_query_chunk_overlap: ran
    • conflicting_numeric_evidence: ran
    • answer_numeric_mismatch: ran

    Limits

    • Findings are heuristic evidence signals, not correctness or contradiction verdicts.
    • Numeric and lexical matching target English; synonyms, conditions and dates need human review.
    • No semantic entailment, source-authority check, unit conversion, currency, token/cost measurement or full SledTrace trace import.
    • No findings does not establish that an answer is supported or correct.
    • The checker is offline; a cloud agent reading its inputs/outputs may send them to its model provider.
    policy-a.md

    Connects securely to your tools. The creator never sees your data.

    About this skill

    A RAG answer looks wrong. Was the evidence empty, repeated, inconsistent, or different from what the answer says?

    Give SledTrace RAG Check the question, retrieved chunks and answer. Your agent runs the included standalone Python checker and explains the findings with exact quotations and JSON field locations. You get a focused investigation plan instead of an unsupported verdict.

    What it checks

    • Empty or whitespace-only retrieved context.
    • Duplicate chunk text after case and whitespace normalization.
    • Low retrieval scores, only with an explicitly supplied threshold and confirmed score direction. Missing scores stay unknown.
    • Weak English query-term overlap, with a clear warning that lexical overlap is not semantic relevance.
    • Candidate numeric disagreements between retrieved chunks.
    • Candidate answer/context numeric mismatches in comparable local wording.

    The numeric rules cover English duration and percentage expressions. They preserve distinctions such as calendar days versus business days, skip certain uncertain or ambiguous clauses, and suppress a mismatch when comparable evidence supports the answer's exact quantity. A conflict among sources can still remain.

    What you receive

    • A concise SKILL.md workflow for evidence collection, execution and interpretation.
    • A self-contained Python checker producing JSON or Markdown reports.
    • Input schema, rule definitions and known limitations.
    • A built-in synthetic demonstration runnable with --demo.

    Requirements and privacy

    Python 3.10+ and an agent with local command/file access. Standard library only: no pip install, API keys, SledTrace server or model downloads. The checker reads one selected JSON input and writes its report to stdout; it makes no network calls and does not modify your application or input file.

    Agensi delivers the skill to your agent. If that agent uses a cloud model, the evidence and report it reads may be processed by that provider. This is an offline checker, not a guarantee that the entire agent session is offline.

    Where it stops

    This is an independent, smaller implementation inspired by SledTrace's diagnostic approach, not the complete SledTrace engine. It does not establish factual truth, source authority or semantic entailment; detect every hallucination; instrument applications; or measure tokens, cost or latency. Numeric findings are candidates requiring checks of dates, conditions and scope. No findings does not prove correctness.

    Input is one normalized JSON case, not arbitrary traces. Limits: 2 MB, 100 chunks and 10,000 characters per text field. Numeric/lexical checks target English. All skipped checks are reported.

    Local validation: 32 automated tests passed on Windows with Python 3.13, including standalone temporary-file execution, exact evidence quotes, missing scores, conflicting sources, elapsed time, malformed JSON and input preservation. Other operating systems and agent integrations have not been independently validated. No general accuracy or improvement claim is made.

    English lexical/numeric heuristics, not a factuality judge. Supports one normalized JSON case; native trace import, real-time capture, dashboard, cost measurement and broad hallucination detection are not included. Conditions, source dates, synonyms and arithmetic require manual review. Input limits: 2 MB, 100 chunks, 10,000 characters per text field. Cloud agents may process evidence through their model provider.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 13 days ago

    Requires

    Python 3.10+
    Terminal access
    Python

    Permissions

    Terminal / Shell
    Read Files
    Write Files

    File Scopes

    *.json
    *.py
    *.md

    Reads only the user-selected JSON case. The agent may create a temporary copy of the bundled script, normalized input and requested report. The checker itself writes only to stdout/stderr, makes no network calls, reads no environment variables and does not modify the input or application. A cloud agent may process evidence through its model provider.

    Listed13 days ago

    What's inside

    Frequently Asked Questions

    No. This skill includes a standalone Python checker. The full SledTrace application is optional and is not used during these checks.

    No. It includes executable Python code, a documented input contract and a workflow that turns the code’s evidence into practical next checks.

    No. It detects specific reproducible signals. Different dates, product tiers, conditions or scopes can explain a numerical disagreement. A clean report is not proof of correctness.

    The checker makes no network calls. Your agent’s provider may process any evidence the agent reads. Only use data you are comfortable sharing with that agent.

    Leave score_threshold absent. The low-score check is opt-in and requires a confirmed higher-is-better scale; missing scores are never treated as zero.

    New in LLM Ops