- Home
- Skills
- Prompt & Skill Engineering
- AI Agent Output Quality Reviewer – Response Evaluation & QA
Works with the AI Agents you already use
AI Agent Output Quality Reviewer – Response Evaluation & QA
Professional QA auditor for AI outputs that identifies hallucinations, scores accuracy, and provides actionable feedback.
$19.99
AI Agent Output Quality Reviewer – Response Evaluation & QA
Example session with this skill installed
Please review this AI-generated summary of a financial report.
Source context: Net profit grew 12% to $4.2M.
AI Output: The company saw a 20% increase in profit, reaching $4.2M.
- Read your context and instructions
- Compiled the ai agent output
I have completed a Full QA Audit of the provided financial summary against the source context. The output contains a significant factual contradiction regarding the growth percentage.
AI Output Quality Review
Output type: Financial Summary
Intended audience: Stakeholders/Investors
Review basis: Full
Overall assessment: Revision required due to critical factual inaccuracy.
1. Executive summary
The response correctly identifies the final profit figure but contains a major hallucination regarding the growth rate (stating 20% instead of the source's 12%). This error undermines the reliability of the entire summary for financial reporting.
2. Quality scorecard
| Dimension | Score | Evidence |
|---|---|---|
| Accuracy and factual support | 1 | Contradicts source: 20% claimed vs 12% in source. |
| Relevance | 5 | Directly addresses the profit figures requested. |
| Clarity | 4 | The sentence structure is clear and professional. |
| Completeness | 3 | Captures the main metric but fails on the delta. |
| Instruction compliance | 2 | Failed to accurately summarize provided data. |
3. Findings
- Severity: Critical
- Excerpt: "The company saw a 20% increase in profit..."
- Problem: Factual hallucination. The source states 12%.
- Why it matters: Inaccurate financial data can lead to regulatory issues or poor investment decisions.
- Recommended correction: Change "20% increase" to "12% increase" to align with source text.
4. Strengths
- Accurately captured the absolute profit figure ($4.2M).
- Maintained a professional tone suitable for the intended audience.
5. Missing requirements
- None identified; the failure is one of accuracy, not omission.
6. Improvement recommendations
- Immediate Fix: Correct the percentage growth figure to 12%.
- Workflow Fix: Implement a numeric cross-check step in the prompt to compare output digits against source digits.
8. Final review status
Revision required: Material issues must be fixed before use.
Next steps
- Correct the growth percentage from 20% to 12%.
- Re-run the review to confirm the revised output is Ready.
- Request a "Rewrite and improve" if you want a corrected version generated now.
Connects securely to your tools. The creator never sees your data.
About this skill
Evaluate AI-generated content against source documents, user instructions, and rigorous quality dimensions. This skill stops hallucinated claims and formatting drift by applying a structured QA framework to agent outputs. It identifies specific defects, calculates severity scores, and provides actionable feedback for developers and product teams who need to ensure their AI outputs are production-ready.
What it does
- Instruction compliance check identifies fulfilled, partially met, or missed requirements from the original prompt.
- Dimensional scoring evaluates accuracy, relevance, clarity, completeness, consistency, and actionability on a 1 to 5 scale.
- Defect inspection flags unsupported claims, internal contradictions, excessive verbosity, and malformed structured output.
- Severity assessment classifies findings as Critical, High, Medium, or Low to prioritize engineering fixes.
- Actionable feedback provides specific excerpts and recommended corrections rather than vague critiques.
How it works
- Supply the AI-generated response along with the original prompt and any relevant source documents.
- The agent establishes a review basis by identifying the target audience and success criteria.
- The agent executes a multi-step audit, scoring dimensions and checking for specific defects.
- Review the structured quality report, including the scorecard, findings, and final status (e.g., Revision Required).
Frameworks & tools
Works with any text or structured output (JSON, Markdown) from LLMs including Claude, GPT-4o, and Gemini. Integrates into manual QA workflows or automated evaluation pipelines.
Why this beats prompting it yourself
Generic "critique this" prompts often suffer from the same sycophancy and hallucinations as the original output. This skill enforces a systematic inspection workflow that separates observable defects from unverifiable claims and uses a grounded scoring rubric to maintain consistency across batch reviews.
Use cases
- RLHF preparation by identifying specific failure modes in agent responses.
- Regression testing to ensure prompt updates do not break formatting or factual accuracy.
- RAG output verification by checking claims against provided context snippets.
- Safety auditing for high-impact financial or legal content that requires disclosure of uncertainty.
Known limitations
Cannot independently verify factual claims without supplied reference materials. Limited assessment possible if the original user prompt is not provided.
Cannot verify facts without provided source documents. Limited compliance checking if the original prompt is missing. Does not monitor live production traffic.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 15 days ago
Permissions
No special permissions declared or detected