More screenshots
Works with the AI Agents you already use
Phoenix: See What Your LLM App Is Actually Doing
Observe LLM apps with Arize Phoenix — OpenTelemetry traces, dataset evals, prompt/model experiments, and production monitoring that catches drift before users do.
$5
Phoenix: See What Your LLM App Is Actually Doing
Example session with this skill installed
Users say our support bot is making things up. Set up Phoenix tracing and evals to find whether it is retrieval, the model, or the prompt — and stop it recurring.
- Read your context and instructions
- Compiled the phoenix
Launched Phoenix locally with px.launch_app() for interactive debugging, then registered a tracer provider and instrumented the OpenAI client and the retriever — two calls, and from that point every LLM call is captured. Opened the prod project, filtered traces by low user-feedback score, and picked a bad one. The trace showed the retriever span returned 3 chunks in 40ms — but the top chunk was from an outdated policy doc. The LLM span showed the model correctly cited the stale chunk. Root cause: retrieval, not generation. Without the trace, the team would have blamed the model and burned a week on prompt tuning. Fixed the document version in the index and added an eval that catches stale citations. Built a dataset of 50–200 representative inputs, versioned it, and ran prompt variants side by side in experiments — code-based checks first (exact match, JSON validity), LLM judges for what code cannot check (faithfulness, tone). Re-running the dataset on every prompt change became the regression suite. In production, judge scores are sampled on a fraction of traffic, embedding-space views watch for query-topic drift, and alerts fire on latency p99, error rate, and judge-score drops — because averages hide the failures and percentiles find them.
Connects securely to your tools. The creator never sees your data.
About this skill
LLM apps fail in ways unit tests cannot catch: the prompt drifted, retrieval returned junk, the model quietly got worse. This skill is the flight recorder: OpenTelemetry tracing where every request is captured with inputs, outputs, token counts, and latency; dataset-based evaluations that replace anecdotes with versioned experiments; LLM-as-judge evaluators plus code-based checks; and production monitoring that watches quality signals, input drift, and percentiles over time. Self-hosted and open source.
It is self-hosted: if you want managed hosting with zero ops, LangSmith or Arize Cloud is the answer.
It observes serving behavior, not training runs — loss curves and hyperparameters belong to Weights & Biases or MLflow.
It is not a model registry or lifecycle governance tool; MLflow owns that job.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
Permissions
No special permissions declared or detected