- Home
- Skills
- Data & Databases
- Data Annotation and Labeling
More screenshots
Works with the AI Agents you already use
Data Annotation and Labeling
Run image, text and audio annotation projects and human evaluation. 10 ordered skills, starting with AI Data Annotation and Labeling, Image and Video Annotation, Text Annotation and NER.
Free
Data Annotation and Labeling
Example session with this skill installed
A small support-software company, "Northwind Desk", wants to label 20,000 customer tickets to train a ticket router with classes billing, bug, how_to, account_access, other. The tool is Label Studio. Deadline: 4 weeks.
- Read your context and instructions
- Compiled the data annotation
Labeling Project Plan: Northwind Desk ticket router
Task rule: Given a ticket, annotator chooses the team that should handle it first
Schema: billing | account_access | bug | how_to | feature_request | other | cannot_determine (definitions, examples, near-misses, tie-break in guideline v1)
Guideline version: v1 (added feature_request, bug-vs-how_to decision order, personal-data flag)
Pilot: 2 rounds x 60 tickets, 3 annotators; Fleiss 0.61 -> 0.72; raw agreement 0.78 shown but flagged as inflated by how_to dominance
Disagreement causes: unclear guideline 9, missing label 4, annotator slip 3, ambiguous 5 -> guideline v1, retraining, example bank
Production QC: 150-item gold set inserted steadily, overlap on a sample, review of low-agreement items, pause below agreed bar after minimum gold count
Adjudication rule: senior reviewer, reason logged, recurring reasons added to guideline
Export: JSONL; raw per-annotator labels kept apart from final; test split frozen before pre-labeling
Datasheet summary: purpose routing; composition English tickets; process as above; limits no non-English, no domain experts; owner support-ops lead
Open risks: personal data redaction not decided; client bar to be confirmed in writing
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
AI data service providers often need a repeatable process, not a one-off prompt. This pack targets: Run image, text and audio annotation projects and human evaluation.
What's inside (10 skills, suggested learning order)
- AI Data Annotation and Labeling — Label data for model training.
- Image and Video Annotation — Segment and tag media.
- Text Annotation and NER — Label entities and intents.
- Audio Annotation and Transcription — Transcribe and label audio.
- Human Evaluation of LLM Responses — Compare and rank model responses.
- Quality Evaluation Rubric Design — Define grading criteria for human and LLM-as-judge raters.
- Evaluation Dataset Creation — Build domain test sets.
- Subject-Matter Expert for AI (SME) — Validate model outputs in technical fields.
- Annotation Team Management — Coordinate annotators and QA.
- Training Data Quality — Assess bias and noise in datasets.
Why a skill pack instead of a one-off prompt
Each skill gives the agent a defined trigger, step-by-step procedure and output format, so results are consistent across sessions and easier to review.
How to use
- Install the pack in your SKILL.md-compatible agent.
- Ask for the task in plain language (for example: "label data for model training").
- Review the output against the skill's quality checks before using it.
Outputs are drafts that need human review; they are not professional, legal or financial advice. Tools, APIs and prices change; check current vendor documentation before relying on version-specific details. Quality depends on the context you give the agent (data, requirements, constraints). Instruction-only skills: they produce documents and plans, not running software.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 8 days ago
- Free with an Agensi account
Permissions
No special permissions declared or detected