Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Data Forge AI Pro

    1

    Provenance-preserving dataset curation, annotation rubrics, leakage controls and reproducible model evaluation.

    $9.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Data Forge AI Pro

    Data Forge AI Pro

    Example session with this skill installed

    Design an evaluation for an AI customer-support agent across refunds, cancellations and escalation, using 250 labeled conversations.

    • Read your context and instructions
    • Compiled the data forge ai

    I have reviewed the task to create a gold evaluation set from 5,000 support tickets. Based on the requirements, I will focus on preventing temporal and entity-based leakage.

    Dataset Evaluation & Split Plan

    ComponentSpecification
    Unit of AnalysisSingle customer support ticket (Thread ID)
    Target LabelsIntent (Refund, Tech Support, Billing, Shipping, Other)
    DeduplicationExact match on Body text; Fuzzy match (95% Jaccard) on metadata-heavy rows
    Split StrategyChronological: Train (First 70%), Val (Next 15%), Test (Final 15%)
    Leakage ControlGroup by Customer_ID to ensure no user appears in both Train and Test
    Gold Set Size500 manually verified records (10% of total)
    MetricsMacro-F1 (imbalance handling), Precision (Billing), Recall (Escalations)

    Provenance & Quality Constraints

    • Observed: 5,000 raw records provided via CSV.
    • Derived: High likelihood of duplicate tickets from follow-up emails.

    Recommendation: Implement a 24-hour window merge for tickets from the same user to prevent near-duplicate leakage.

    Privacy Check: FAILED - Raw Email and Phone columns detected. De-identification step required before annotation.

    Annotation Rubric Preview

    • Label: Refund_Request
    • Definition: User explicitly asks for money back or disputes a charge.
    • Edge Case: "I want to cancel" without mentioning money is Cancellation, not Refund.
    • Conflict Resolution: Tie-break by Senior Annotator if confidence < 0.8.

    Next steps

    1. Approve the chronological split strategy to account for shifting support topics.
    2. Confirm the PII masking logic for the Email and Phone fields.
    3. Provide 20 sample rows to initialize the annotation decision tree.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Trustworthy datasets and evaluations, not fabricated benchmark wins. Built for ML engineers, data teams and AI product evaluators. Includes a specialist SKILL.md, four operating modes, a 10-stage expert checklist, domain stop rules, a reusable delivery template, an evaluation rubric and practical acceptance prompts. Runs as a portable instruction skill in compatible hosts; host tool permissions and live integrations are not included.

    Does not include standalone SaaS, provider API access, or preconfigured third-party integrations. Real host performance not yet independently tested.

    Does not execute code or browse the web unless tools are provided. Requires raw data samples to perform audits. Cannot verify legal ownership of third-party data.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    Permissions

    Read Files
    Listed24 days ago

    What's inside

    Frequently Asked Questions

    DataForge AI Pro identifies hidden risks like PII, class imbalances, and data leakage where training samples accidentally bleed into test sets, preventing artificially inflated performance metrics.

    The skill is framework-agnostic and designed to integrate with Python-based workflows (Pandas, Polars), Hugging Face Datasets, and popular evaluation tools like Ragas or G-Eval.

    You receive the core DataForge AI Pro skill logic, the 10-point expert checkpoint system for data auditing, and standardized templates for annotation rubrics and evaluation harnesses.

    DataForge AI Pro is a logic and strategy engine; it provides the systematic framework and transformation thresholds, but you must grant it access to your specific data processing tools to execute cleaning.

    Yes, it is specifically designed for RAG evaluation, offering a structured approach to building adversarial test cases and multi-dimensional scoring rubrics for groundedness and safety.

    Setup involves importing the skill into your preferred AI agent environment and providing the initial schema or raw data profile for the skill to begin its 10-point audit process.

    New in LLM Ops