Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    ADK-EvalGate: Agent CI Regression & Provenance Harness

    1

    Block silent prompt, model and tool-schema regressions in CI with deterministic Google ADK trajectory checks, injection probes and SHA-256 provenance.

    $49

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    ADK-EvalGate: Agent CI Regression & Provenance Harness

    ADK-EvalGate: Agent CI Regression & Provenance Harness

    Example session with this skill installed

    Gate candidate ADK trace run_pr184.jsonl against baseline run_v2_4.jsonl and golden_suite.json with --strict-exit for our pull request.

    • Read your context and instructions
    • Compiled the adk-evalgate

    Exit code 1 (BLOCK): 3 of 5 golden trajectories regressed—catching refund_order(amount_cents='true') JSON Schema type failure, a 4x identical search_kb retry loop (+68% token spike), and leaked injection marker CANARY_7F9A—with SHA-256 provenance scorecard and PR comment Markdown.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    A minor prompt tweak or model version bump can silently break tool-call JSON types, spin an agent into a 4x retry loop, or let an untrusted document exfiltrate a canary secret—long before manual spot-checks notice.

    What your AI assistant does with it

    • ADK & JSONL trajectory replay: Evaluates Google ADK session event dumps and flat traces against golden tool sequences, forbidden tools and step/cost caps.
    • Strict tool-arg schema gate: Validates every function_call payload against JSON Schema—catching bool-as-int confusion, missing fields and bad enums.
    • Untrusted-input canary audit: Verifies adversarial web or email tool outputs never leak canary markers into outbound tool arguments or logs.
    • Tamper-evident CI provenance: Scrubs leaked API keys from traces, exits non-zero on regressions and emits a SHA-256 signed scorecard for PRs.

    How a run works

    1. Define your golden cases, tool JSON Schemas and regression policy thresholds
    2. Capture baseline and candidate Google ADK or agent trajectory JSON/JSONL logs
    3. Run adk_eval_gate.py --strict-exit locally or inside GitHub Actions
    4. Review the PR regression table, schema errors and hashed provenance scorecard

    Built for: Heads of AI & Platform leads, Staff ML & Agent engineers, Google ADK & MCP builders.

    Included: Skill instructions with baseline-promotion STOP AND VERIFY rules, adk_eval_gate.py: ADK trajectory validator, JSON-Schema checker & canary gate, PR Markdown diff, SpreadsheetML workbook & offline HTML CI dashboard, Google ADK 2.x event-trace & GitHub Actions CI integration reference, 5-case regression evaluation suite.

    Part of the B2B Operations & FinOps suite by Production AI Agents. Every calculation runs locally in standard-library Python (Decimal math, zero hidden network calls) and pauses at a human STOP AND VERIFY gate before any external write or sign-off.

    Deterministic trajectory, schema and canary regression harness; use agent-evals for Wilson/bootstrap statistical intervals on open-ended prose. Always review generated calculations, drafts and SQL/API payloads before executing external actions.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed today

    Requires

    Python 3.10+
    Python

    Permissions

    File Scopes

    Golden
    eval
    suite
    JSON
    and
    baseline/candidate
    ADK
    or
    JSONL
    traces
    (read);
    CI
    regression
    reports
    Excel
    XML
    SHA-256
    scorecard
    (write)

    adk_eval_gate.py runs 100% deterministically offline with zero LLM API calls and scrubs leaked API keys/bearer tokens before writing reports.

    Listedtoday

    What's inside

    Frequently Asked Questions

    No. All schema, trajectory ordering, retry-loop, cost-delta and canary-exfiltration checks run deterministically in Python stdlib with zero API spend and zero flaky CI verdicts.

    ADK-EvalGate owns deterministic trajectory replay, strict JSON-Schema tool-arg validation, canary leak checks and SHA-256 CI provenance for Google ADK and JSONL traces, complementing statistical Wilson/bootstrap intervals.

    adk_eval_gate.py automatically scrubs sk-*, AIza*, ghp_* and Bearer tokens before writing JSON, Markdown, XML or HTML build artifacts.

    New in LLM Ops