More screenshots
Works with the AI Agents you already use
Research Evaluation Designer
Design an evidence-backed evaluation rubric for LLM, RAG, research, or agent outputs.
$19.99
Research Evaluation Designer
Example session with this skill installed
Design a HYBRID evaluation rubric for a clinical-QA RAG system in the medical domain. Include the RAGAS core, domain-specific clinical safety dimensions, anchored human review, a HIPAA-related evaluation overlay, and a 20–30 example pilot. Do not present any threshold as production-validated without pilot evidence.
- Read your context and instructions
- Compiled the research evaluation designer
Clinical-QA RAG Evaluation Blueprint
System: clinical-QA RAG pipeline | Mode: HYBRID | Domain: medical
| Dimension | Method | Initial threshold | Evidence status |
|---|---|---|---|
| Faithfulness | Claim extraction plus NLI against retrieved context | 0.80 | Pilot target, not production validated |
| Answer relevance | Reverse-question generation plus cosine similarity | 0.80 | Pilot target |
| Context precision | Useful chunks divided by total retrieved chunks | 0.70 | Pilot target |
| Context recall | Ground-truth statements covered divided by total statements | 0.80 | Pilot target |
| Clinical accuracy | Claims checked against a curated medical-guidelines set | 0.85 | Domain-specific proposed metric |
| Contraindication detection | Dangerous recommendations flagged across seeded cases | 0.90 | Requires explicit pilot evidence before adoption |
Human anchors
Coherence: 1 = contradictory or illogical; 3 = mostly coherent with minor gaps; 5 = fully coherent clinical reasoning.
Completeness: 1 = misses most sub-questions; 3 = addresses the main question only; 5 = addresses every stated sub-question.
Compliance overlay
For the supplied HIPAA requirement, verify that the evaluation set contains no patient identifiers and retain provenance for clinical claims. This mapping supports evaluation design; it does not establish legal compliance.
Pilot
Use 20–30 stratified examples including retrieval misses, irrelevant context, contraindication traps, and complete answers. Double-rate a subset, examine disagreement, revise anchors, and recalibrate thresholds before any production gate.
Connects securely to your tools. The creator never sees your data.
About this skill
Design a domain-specific evaluation rubric for LLM, RAG, research, or multi-agent outputs. Research Evaluation Designer selects evidence-backed dimensions, combines automated metrics with anchored human scales, adds compliance mappings only when supplied, rejects unsupported thresholds, and includes a calibration plan without pretending the rubric is already validated.
You receive a dimension table, measurement methods, source register, human-rating anchors, compliance overlay, threshold status, and a pilot plan. The package includes the full SKILL.md workflow, RAGAS references, calibration guidance, compliance matrix, JSON rubric schema, deterministic checker, worked sample, tests, and a six-page PDF buyer guide.
It designs rubrics; it does not run evaluations, generate a dataset, establish regulatory compliance, or claim production validity without a pilot.
Designs evaluation rubrics only. It does not run evaluations, generate datasets, establish regulatory compliance, or claim production validity. Thresholds remain pilot targets until supported by calibration evidence.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 2 months ago
Permissions
No special permissions declared or detected