Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Context Fidelity Preflight — Prompt & RAG QA

    1

    See what a prompt or context change loses. Compare the same cases for missing facts, citations and output fields.

    Free

    1 useSecurity scanned
    Context Fidelity Preflight — Prompt & RAG QA

    Context Fidelity Preflight — Prompt & RAG QA

    Example session with this skill installed

    Synthetic offline example using a deterministic retrieval/answer test double, not an LLM. Check whether a proposed context-pruning rule loses a required source condition. Keep the actual-model behavior gate unevaluated.

    Frozen cache policy

    • [cache-1] Reuse build cache for source-only changes.
    • [cache-2] If compiler flags change, discard the cache before building.
    • [cache-3] Keep a failed build log for seven days.

    The baseline retains all three passages. The candidate retains cache-1 only. Before running it, require: source-only question → reuse with cache-1; compiler-flags question → discard with cache-2; release-approver question → unknown with no invented citation.

    Run three deterministic observations per case/version. If a required passage is lost, demonstrate the smallest retention repair and rerun every case sharing that pruning rule. Report model/provider, token, cost and latency evidence as unavailable rather than inferring it.

    • Read your context and instructions
    • Compiled the context fidelity preflight

    Context coverage: repair required

    Actual offline test-double execution. This comparison used a fixed three-passage corpus and predeclared checks; it made no model or provider calls.

    CaseBaselineCandidate retaining cache-1 onlyAfter retaining cache-1 and cache-2
    Source-only changeReuse; cites cache-1 · 3/3Reuse; cites cache-1 · 3/3Reuse; cites cache-1 · 3/3
    Compiler flags changedDiscard; cites cache-2 · 3/3Unknown; no citation · 0/3Discard; cites cache-2 · 3/3
    Who approved the release?Unknown; no citation · 3/3Unknown; no citation · 3/3Unknown; no citation · 3/3
    Accepted observations9/96/99/9

    Lost condition: cache-policy.md:3 says to discard the cache when compiler flags change. The candidate removed its anchor, cache-2. Runs candidate/compiler-flags/1, /2 and /3 all failed that predeclared requirement.

    Repair: retain cache-2 alongside cache-1. All three cases were rerun because they share the pruning rule. The frozen corpus contained no approver claim; the negative-evidence check found no approving passage, so the uncertainty response remained required.

    The candidate's source-coverage result was REVISE. After repair, source coverage passed within this synthetic harness. These 27 deterministic executions are not a stochastic-model repeatability test.

    Actual model behavior: HOLD / not run. No model release, provider, tokens, cost, latency or production retrieval result was measured. Next, run the same versioned cases through the approved real evaluation runner before making a model-behavior release decision.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    A shorter context can remove the one exception that makes an answer correct. The ordinary question still works; the edge case loses its source.

    Context Fidelity Preflight helps you review a prompt, retrieval or context-pruning change before release. Bring the baseline, candidate, source corpus and your authorized evaluation runner. Your agent builds versioned cases around the facts, citations, uncertainty and output fields that must survive the change.

    It compares the same cases under comparable model settings and a declared repeat policy. Answerable questions need precise source passages. Questions the corpus cannot answer need a checked source boundary and an expected uncertainty response.

    The result is a specific decision

    You get the failing cases and run IDs, the fact or citation lost, and the smallest repair or next observation needed. Cost and latency enter the comparison only when the runner measured them. Missing runs or required evidence keep the model-behavior decision on HOLD / not run.

    This is an instruction-only evaluation workflow. Use your own approved provider or harness; no model service, key or evaluation platform is bundled. Its conclusion covers the tested cases and settings, with missing measurements left unknown.

    The default three paired runs are a small repeatability check, not statistical significance. Human or calibrated semantic review may be needed. A verdict applies only to tested fixtures, model, settings, corpus, and time window.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    1 use

    Times used to date

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    • Free with an Agensi account

    Permissions

    Read Files
    Write Files
    Terminal / Shell
    Network Access

    Read evaluation files and approved source material; write fixtures and receipts when requested. Use local shell or provider calls only through an authorized harness. Do not transmit private source material outside its approved boundary.

    Listed1 month ago
    Updated24 days ago

    What's inside

    Frequently Asked Questions

    The baseline and candidate, frozen source corpus and cases, exact model release and settings, and an authorized runner. Required paired observations and applicable source evidence must exist; otherwise that verdict stays HOLD / not run.

    No. It demonstrates a specific source passage lost by pruning and a repair in a deterministic offline test double. No LLM or provider was called. The actual-model gate remains unevaluated.

    It can compare recorded tokens, cost with its pricing basis and latency from comparable runs. Missing measurements remain unknown; shorter text alone is not measured savings.

    For stochastic output, the default is at least three baseline and three candidate runs per fixture under the same declared policy. That small repeatability check is not statistical significance or a production guarantee.

    New in LLM Ops