More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Data Annotation and Labeling

    1

    Run image, text and audio annotation projects and human evaluation. 10 ordered skills, starting with AI Data Annotation and Labeling, Image and Video Annotation, Text Annotation and NER.

    Free

    Security scanned
    Data Annotation and Labeling

    Data Annotation and Labeling

    Example session with this skill installed

    A small support-software company, "Northwind Desk", wants to label 20,000 customer tickets to train a ticket router with classes billing, bug, how_to, account_access, other. The tool is Label Studio. Deadline: 4 weeks.

    • Read your context and instructions
    • Compiled the data annotation

    Labeling Project Plan: Northwind Desk ticket router

    Task rule: Given a ticket, annotator chooses the team that should handle it first
    Schema: billing | account_access | bug | how_to | feature_request | other | cannot_determine (definitions, examples, near-misses, tie-break in guideline v1)
    Guideline version: v1 (added feature_request, bug-vs-how_to decision order, personal-data flag)
    Pilot: 2 rounds x 60 tickets, 3 annotators; Fleiss 0.61 -> 0.72; raw agreement 0.78 shown but flagged as inflated by how_to dominance
    Disagreement causes: unclear guideline 9, missing label 4, annotator slip 3, ambiguous 5 -> guideline v1, retraining, example bank
    Production QC: 150-item gold set inserted steadily, overlap on a sample, review of low-agreement items, pause below agreed bar after minimum gold count
    Adjudication rule: senior reviewer, reason logged, recurring reasons added to guideline
    Export: JSONL; raw per-annotator labels kept apart from final; test split frozen before pre-labeling
    Datasheet summary: purpose routing; composition English tickets; process as above; limits no non-English, no domain experts; owner support-ops lead
    Open risks: personal data redaction not decided; client bar to be confirmed in writing

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    AI data service providers often need a repeatable process, not a one-off prompt. This pack targets: Run image, text and audio annotation projects and human evaluation.

    What's inside (10 skills, suggested learning order)

    1. AI Data Annotation and Labeling — Label data for model training.
    2. Image and Video Annotation — Segment and tag media.
    3. Text Annotation and NER — Label entities and intents.
    4. Audio Annotation and Transcription — Transcribe and label audio.
    5. Human Evaluation of LLM Responses — Compare and rank model responses.
    6. Quality Evaluation Rubric Design — Define grading criteria for human and LLM-as-judge raters.
    7. Evaluation Dataset Creation — Build domain test sets.
    8. Subject-Matter Expert for AI (SME) — Validate model outputs in technical fields.
    9. Annotation Team Management — Coordinate annotators and QA.
    10. Training Data Quality — Assess bias and noise in datasets.

    Why a skill pack instead of a one-off prompt

    Each skill gives the agent a defined trigger, step-by-step procedure and output format, so results are consistent across sessions and easier to review.

    How to use

    1. Install the pack in your SKILL.md-compatible agent.
    2. Ask for the task in plain language (for example: "label data for model training").
    3. Review the output against the skill's quality checks before using it.

    Outputs are drafts that need human review; they are not professional, legal or financial advice. Tools, APIs and prices change; check current vendor documentation before relying on version-specific details. Quality depends on the context you give the agent (data, requirements, constraints). Instruction-only skills: they produce documents and plans, not running software.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 8 days ago

    • Free with an Agensi account

    Permissions

    No special permissions declared or detected

    Listed8 days ago

    What's inside

    Frequently Asked Questions

    10 skills, each with SKILL.md and references/sources.md: AI Data Annotation and Labeling; Image and Video Annotation; Text Annotation and NER; Audio Annotation and Transcription; Human Evaluation of LLM Responses; Quality Evaluation Rubric Design; Evaluation Dataset Creation; Subject-Matter Expert for AI (SME); Annotation Team Management; Training Data Quality.

    AI data service providers.

    Agents that read the SKILL.md format. See the Compatibility note for what was actually tested.

    New in Data & Databases