More screenshots
Works with the AI Agents you already use
Agent Reliability Pack — Find Where Your Agent Fails
Reads your run traces and reports task success, recovery, retry amplification, loops and the top failure classes, each with the fix that actually applies to it.
$19
Agent Reliability Pack — Find Where Your Agent Fails
Example session with this skill installed
python3 scripts/reliability_scan.py traces.jsonl --report reliability-report.md
Trace rows
{"run_id":"run-031","step":2,"tool":"search","status":"error","error":"timeout","retry":true,"latency_ms":30000}
- Read your context and instructions
- Compiled the agent reliability
agent-reliability-pack 56 runs · 258 steps
success 58.9% recovery 78.3% retry amp 1.13× loops 2
findings 10 high 4 medium 5 low 1
HIGH R1 success rate 58.9% of 56 runs ended without an error step
HIGH R3 loop run-053: search called 3× with the same arguments
HIGH R4 timeout 8 errors · search
MEDIUM R11 retry amplification 26 retries for 23 errors (1.1×)
MEDIUM R12 unbounded run run-055: 45 steps (cap 40)
MEDIUM R5 rate limit 5 errors · model
FAIL findings at or above 'high'
report reliability-report.md
What to do next
- Fix the timeout first: it is 35% of the errors and it has a retry path already.
- Add a repeated-action cap; both loop runs completed without an error, so nothing flagged them.
- Retry only transient classes, and keep one retry owner rather than two layers each retrying.
- Cap steps per run at the orchestrator, not in the prompt.
- Watch recovery after each change; it should rise before success does.
Connects securely to your tools. The creator never sees your data.
About this skill
A demo proves an agent can finish once. Production asks whether it finishes after a timeout, a duplicate event, a tool outage or a model change — and your traces already contain the answer. This reads them and turns a pile of JSONL into a ranked reliability review.
It classifies every failed step into a named class, then measures the things that actually separate a flaky agent from a broken one:
- Task success and recovery rate, reported together so a retry policy cannot hide a failure
- Retry amplification, which catches retrying errors that will never change
- Loops: the same tool called three times with identical arguments, with the run and the arguments
- Unbounded runs that pass the step ceiling, the denial-of-wallet failure
- Failure counts by class and by tool, so the fix goes to the right owner
- Latency p50 and p95 alongside the failures, because a timeout that "recovers" still ships slowly
Each finding pairs the diagnosis with the fix that fits it: retry transient classes only, return tool errors as tool messages so the model can adapt, repair a schema error once and escalate, compact instead of retrying a context overflow, cap loops in the orchestrator rather than asking the model to stop.
Runs on the standard library. Point it at traces, get a Markdown report and JSON, and gate CI with --fail-on high.
Good to know: the trace says a run ended without an error step; it does not say the answer was right. Recovery is inferred from a later non-error step, not a verified final state, and error classification is keyword-based on your error field, so normalise your codes before relying on the counts. This is a diagnosis tool, not an eval — pair it with outcome evals before quoting a success rate. The shipped sample is deliberately failure-rich: 56 runs, 23 errors, 58.9% success, 2 loops.
Success is inferred from the absence of error steps, not output correctness. Recovery is inferred from later non-error steps rather than verified final state. Requires JSONL input.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 1 day ago
Requires