Works with the AI Agents you already use
Context Fidelity Preflight — Prompt & RAG QA
See what a prompt or context change loses. Compare the same cases for missing facts, citations and output fields.
Free
Context Fidelity Preflight — Prompt & RAG QA
Example session with this skill installed
Synthetic offline example using a deterministic retrieval/answer test double, not an LLM. Check whether a proposed context-pruning rule loses a required source condition. Keep the actual-model behavior gate unevaluated.
Frozen cache policy
[cache-1]Reuse build cache for source-only changes.[cache-2]If compiler flags change, discard the cache before building.[cache-3]Keep a failed build log for seven days.
The baseline retains all three passages. The candidate retains cache-1 only. Before running it, require: source-only question → reuse with cache-1; compiler-flags question → discard with cache-2; release-approver question → unknown with no invented citation.
Run three deterministic observations per case/version. If a required passage is lost, demonstrate the smallest retention repair and rerun every case sharing that pruning rule. Report model/provider, token, cost and latency evidence as unavailable rather than inferring it.
- Read your context and instructions
- Compiled the context fidelity preflight
Context coverage: repair required
Actual offline test-double execution. This comparison used a fixed three-passage corpus and predeclared checks; it made no model or provider calls.
| Case | Baseline | Candidate retaining cache-1 only | After retaining cache-1 and cache-2 |
|---|---|---|---|
| Source-only change | Reuse; cites cache-1 · 3/3 | Reuse; cites cache-1 · 3/3 | Reuse; cites cache-1 · 3/3 |
| Compiler flags changed | Discard; cites cache-2 · 3/3 | Unknown; no citation · 0/3 | Discard; cites cache-2 · 3/3 |
| Who approved the release? | Unknown; no citation · 3/3 | Unknown; no citation · 3/3 | Unknown; no citation · 3/3 |
| Accepted observations | 9/9 | 6/9 | 9/9 |
Lost condition: cache-policy.md:3 says to discard the cache when compiler flags change. The candidate removed its anchor, cache-2. Runs candidate/compiler-flags/1, /2 and /3 all failed that predeclared requirement.
Repair: retain cache-2 alongside cache-1. All three cases were rerun because they share the pruning rule. The frozen corpus contained no approver claim; the negative-evidence check found no approving passage, so the uncertainty response remained required.
The candidate's source-coverage result was REVISE. After repair, source coverage passed within this synthetic harness. These 27 deterministic executions are not a stochastic-model repeatability test.
Actual model behavior: HOLD / not run. No model release, provider, tokens, cost, latency or production retrieval result was measured. Next, run the same versioned cases through the approved real evaluation runner before making a model-behavior release decision.
Connects securely to your tools. The creator never sees your data.
About this skill
A shorter context can remove the one exception that makes an answer correct. The ordinary question still works; the edge case loses its source.
Context Fidelity Preflight helps you review a prompt, retrieval or context-pruning change before release. Bring the baseline, candidate, source corpus and your authorized evaluation runner. Your agent builds versioned cases around the facts, citations, uncertainty and output fields that must survive the change.
It compares the same cases under comparable model settings and a declared repeat policy. Answerable questions need precise source passages. Questions the corpus cannot answer need a checked source boundary and an expected uncertainty response.
The result is a specific decision
You get the failing cases and run IDs, the fact or citation lost, and the smallest repair or next observation needed. Cost and latency enter the comparison only when the runner measured them. Missing runs or required evidence keep the model-behavior decision on HOLD / not run.
This is an instruction-only evaluation workflow. Use your own approved provider or harness; no model service, key or evaluation platform is bundled. Its conclusion covers the tested cases and settings, with missing measurements left unknown.
The default three paired runs are a small repeatability check, not statistical significance. Human or calibrated semantic review may be needed. A verdict applies only to tested fixtures, model, settings, corpus, and time window.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
1 use
Times used to date
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 24 days ago
- Free with an Agensi account
Permissions
Read evaluation files and approved source material; write fixtures and receipts when requested. Use local shell or provider calls only through an authorized harness. Do not transmit private source material outside its approved boundary.