- Home
- Skills
- Testing & Debugging
- Agent Tool Change Regression Builder
Agent Tool Change Regression Builder
Turn an agent tool upgrade into traceable offline regression cases for changed schemas, approval scope and uncertain retries.
Free
Works with the AI Agents you already use
Agent Tool Change Regression Builder
Example session with this skill installed
Use the included fictional draft-to-send.json comparison to generate a complete offline regression plan. Run only the included fake-gateway simulator in buggy and fixed modes, then check each observation envelope. Return the complete plan and both checker reports. Do not run a real agent, call external tools or send any messages.
- Read your context and instructions
- Compiled the agent tool change
Local synthetic demonstration
The builder generated 10 change records and 65 proposed cases, including 9 bounded compatibility counterexamples. The fictional buggy gateway produced 4 failing approval/retry probes among 12 recorded probes. The corrected stub's 12 recorded probes passed. Schema and manual-review cases remain proposals; this is not an overall regression-suite or real-agent pass.
No real mail adapter, agent, recipient, network call or Agensi execution was involved. All three complete generated outputs follow.
Complete generated plan
{
"format_version": 1,
"builder_version": "1.0.0",
"execution": "offline_proposal_only",
"snapshots": {
"old": {
"label": "fictional-v1",
"sha256": "07388b27b2e5e8a2572e9022f9e3eca8b46ff00118e7de002822cd7ea54ca3ca"
},
"new": {
"label": "fictional-v2",
"sha256": "58ca5d6cdda4dbd7e3164ff2e7529cf805e755da089521245a3d0ca4e54717a1"
}
},
"summary": {
"changes": 10,
"cases": 65,
"compatibility_counterexamples": 9
},
"changes": [
{
"path": "/contract/approval",
"old_present": true,
"new_present": true,
"old": "preapproved",
"new": "per_action",
"tool": "mail.compose",
"id": "CHG-0ada0339e9c2"
},
{
"path": "/contract/idempotency",
"old_present": true,
"new_present": true,
"old": "none",
"new": "key",
"tool": "mail.compose",
"id": "CHG-db016ae9cdf3"
},
{
"path": "/contract/idempotency_key",
"old_present": false,
"new_present": true,
"old": null,
"new": "request_id",
"tool": "mail.compose",
"id": "CHG-e629806a8fd9"
},
{
"path": "/contract/notes",
"old_present": true,
"new_present": true,
"old": "Fictional version 1 creates drafts only.",
"new": "Fictional version 2 sends by default. Sandbox only.",
"tool": "mail.compose",
"id": "CHG-bb35a93560d3"
},
{
"path": "/contract/retry",
"old_present": true,
"new_present": true,
"old": "never",
"new": "after_verification",
"tool": "mail.compose",
"id": "CHG-bad2765fb96b"
},
{
"path": "/contract/side_effect",
"old_present": true,
"new_present": true,
"old": "write",
"new": "send",
"tool": "mail.compose",
"id": "CHG-f8ab8cb1d1d1"
},
{
"path": "/input_schema/properties/mode/default",
"old_present": true,
"new_present": true,
"old": "draft",
"new": "send",
"tool": "mail.compose",
"id": "CHG-1c18329ebc3f"
},
{
"path": "/input_schema/properties/request_id",
"old_present": false,
"new_present": true,
"old": null,
"new": {
"type": "string",
"minLength": 1
},
"tool": "mail.compose",
"id": "CHG-13ffe81318ce"
},
{
"path": "/input_schema/required",
"old_present": true,
"new_present": true,
"old": [
"recipient",
"body"
],
"new": [
"recipient",
"body",
"request_id"
],
"tool": "mail.compose",
"id": "CHG-838c0025981c"
},
{
"path": "/output_schema/properties/state/enum",
"old_present": true,
"new_present": true,
"old": [
"drafted"
],
"new": [
"queued",
"sent"
],
"tool": "mail.compose",
"id": "CHG-398a6954e55c"
}
],
"cases": [
{
"tool": "mail.compose",
"kind": "manual_change_review",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"assertions": [
"Review linked changes with contract owner; probes are not proof."
],
"id": "CASE-76fc4ce86df7"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:baseline",
"value": {
"body": "",
"mode": "draft",
"recipient": "x"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-c8da17457e19"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:wrong_type",
"value": null,
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-44c0f43c5787"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:omit:body",
"value": {
"mode": "draft",
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-99f122c25ff4"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:body/wrong_type",
"value": {
"body": null,
"mode": "draft",
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-c31b73bc463a"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:body/length_1",
"value": {
"body": "x",
"mode": "draft",
"recipient": "x"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-990d978286bf"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:body/length_2",
"value": {
"body": "xx",
"mode": "draft",
"recipient": "x"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-3ba32b507890"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:omit:mode",
"value": {
"body": "",
"recipient": "x"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-09fb4d6591b6"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:mode/wrong_type",
"value": {
"body": "",
"mode": null,
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-ba5e4a7093e5"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:mode/enum_1",
"value": {
"body": "",
"mode": "send",
"recipient": "x"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-7850b81a395c"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:mode/outside_enum",
"value": {
"body": "",
"mode": "__outside_enum__",
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-70ff3bfb9678"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:mode/length_0",
"value": {
"body": "",
"mode": "",
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-e87209b7f752"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:mode/length_1",
"value": {
"body": "",
"mode": "x",
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-b4afd6608ae6"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:mode/length_2",
"value": {
"body": "",
"mode": "xx",
"recipient": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-963f8caf45df"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:omit:recipient",
"value": {
"body": "",
"mode": "draft"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-5d0cebf7d81a"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:recipient/wrong_type",
"value": {
"body": "",
"mode": "draft",
"recipient": null
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-180651625079"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:recipient/length_0",
"value": {
"body": "",
"mode": "draft",
"recipient": ""
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-8fb5716d7ad6"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:recipient/length_2",
"value": {
"body": "",
"mode": "draft",
"recipient": "xx"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-d8e023d04c07"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:extra_property",
"value": {
"body": "",
"mode": "draft",
"recipient": "x",
"__unexpected__": true
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-84afcce2998c"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "old:fixture:draft-alex",
"value": {
"recipient": "alex@example.invalid",
"body": "Fictional project update"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": true,
"id": "CASE-4f895bb07044"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:baseline",
"value": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-d34a313d165c"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:omit:body",
"value": {
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-df1ddead1448"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:body/wrong_type",
"value": {
"body": null,
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-5aef580ef40c"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:body/length_1",
"value": {
"body": "x",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-84398daa9edb"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:body/length_2",
"value": {
"body": "xx",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-05c001f255dd"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:omit:mode",
"value": {
"body": "",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-31f459efcc9d"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:mode/wrong_type",
"value": {
"body": "",
"mode": null,
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-d0cd6fccdb79"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:mode/enum_1",
"value": {
"body": "",
"mode": "send",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-825841c5455c"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:mode/outside_enum",
"value": {
"body": "",
"mode": "__outside_enum__",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-35ea7a82df72"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:mode/length_0",
"value": {
"body": "",
"mode": "",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-be969d5407a3"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:mode/length_1",
"value": {
"body": "",
"mode": "x",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-58bc67ef4db3"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:mode/length_2",
"value": {
"body": "",
"mode": "xx",
"recipient": "x",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-f94fe3c61dca"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:omit:recipient",
"value": {
"body": "",
"mode": "draft",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-78d9acab8170"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:recipient/wrong_type",
"value": {
"body": "",
"mode": "draft",
"recipient": null,
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-53faae24da43"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:recipient/length_0",
"value": {
"body": "",
"mode": "draft",
"recipient": "",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-108b22b65978"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:recipient/length_2",
"value": {
"body": "",
"mode": "draft",
"recipient": "xx",
"request_id": "x"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-a5144bed6c05"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:request_id/wrong_type",
"value": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": null
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-255780402990"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:request_id/length_0",
"value": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": ""
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-45f5e9844267"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:request_id/length_2",
"value": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "xx"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-48166357c3d0"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:extra_property",
"value": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x",
"__unexpected__": true
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-5116d716a0b1"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c"
],
"status": "proposed_not_executed",
"schema": "input_schema",
"origin": "new:fixture:draft-alex",
"value": {
"recipient": "alex@example.invalid",
"body": "Fictional project update",
"request_id": "fictional-001"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": false,
"id": "CASE-ba2537102aa0"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:baseline",
"value": {
"state": "drafted"
},
"expected_old_valid": true,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-cf61a7606ab4"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:wrong_type",
"value": null,
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-d72a78e23565"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:omit:state",
"value": {},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-c20fbd5d63e7"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:state/wrong_type",
"value": {
"state": null
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-655ffbf15816"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:state/outside_enum",
"value": {
"state": "__outside_enum__"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-e169ea24ea45"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:state/length_0",
"value": {
"state": ""
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-30913cd5ba8c"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:state/length_1",
"value": {
"state": "x"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-5eef59a6a95d"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:state/length_2",
"value": {
"state": "xx"
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-ec9bf28c9ac5"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "old:extra_property",
"value": {
"state": "drafted",
"__unexpected__": true
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-f3ad0f6a78e9"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "new:baseline",
"value": {
"state": "queued"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": true,
"id": "CASE-de1085c29a11"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "new:state/enum_1",
"value": {
"state": "sent"
},
"expected_old_valid": false,
"expected_new_valid": true,
"compatibility_counterexample": true,
"id": "CASE-27aaa7ccd775"
},
{
"tool": "mail.compose",
"kind": "schema_probe",
"change_ids": [
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"schema": "output_schema",
"origin": "new:extra_property",
"value": {
"state": "queued",
"__unexpected__": true
},
"expected_old_valid": false,
"expected_new_valid": false,
"compatibility_counterexample": false,
"id": "CASE-cad5a6171166"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "matching",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "allow_under_supplied_contract_only",
"max_attempted_calls": 1,
"id": "CASE-62fecbef4ace"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "missing",
"scope": "matching",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_approval",
"max_attempted_calls": 0,
"id": "CASE-2ed985772898"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "revoked",
"scope": "matching",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_approval",
"max_attempted_calls": 0,
"id": "CASE-f6d8d3bca6f6"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "expired",
"scope": "matching",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_approval",
"max_attempted_calls": 0,
"id": "CASE-f625b8d7da11"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "different_recipient",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_authorization",
"max_attempted_calls": 0,
"id": "CASE-84aa961097a2"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "different_data",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_authorization",
"max_attempted_calls": 0,
"id": "CASE-8accdda57e98"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "different_amount",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_authorization",
"max_attempted_calls": 0,
"id": "CASE-e96a4c4eff2d"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "different_action",
"previous_outcome": "not_started",
"approval_from_previous_attempt": false
},
"expected_decision": "require_current_authorization",
"max_attempted_calls": 0,
"id": "CASE-577c8bbe3f21"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "matching",
"previous_outcome": "unknown",
"approval_from_previous_attempt": true
},
"expected_decision": "verify_outcome_before_decision",
"max_attempted_calls": 0,
"id": "CASE-7197f4a5ab60"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "matching",
"previous_outcome": "succeeded",
"approval_from_previous_attempt": true
},
"expected_decision": "stop_already_succeeded",
"max_attempted_calls": 0,
"id": "CASE-7efa9293ec6b"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "matching",
"previous_outcome": "failed_no_effect",
"approval_from_previous_attempt": true
},
"expected_decision": "require_fresh_per_action_approval",
"max_attempted_calls": 0,
"id": "CASE-69ccdbd040c2"
},
{
"tool": "mail.compose",
"kind": "approval_retry_probe",
"change_ids": [
"CHG-0ada0339e9c2",
"CHG-db016ae9cdf3",
"CHG-e629806a8fd9",
"CHG-bb35a93560d3",
"CHG-bad2765fb96b",
"CHG-f8ab8cb1d1d1",
"CHG-1c18329ebc3f",
"CHG-13ffe81318ce",
"CHG-838c0025981c",
"CHG-398a6954e55c"
],
"status": "proposed_not_executed",
"arguments": {
"body": "",
"mode": "draft",
"recipient": "x",
"request_id": "x"
},
"context": {
"approval": "valid",
"scope": "matching",
"previous_outcome": "not_started",
"approval_from_previous_attempt": true
},
"expected_decision": "require_fresh_per_action_approval",
"max_attempted_calls": 0,
"id": "CASE-f35db73095c4"
}
]
}
Complete buggy-stub observation check
{
"format_version": 1,
"status": "fail",
"scope": "supplied_offline_decisions_only",
"plan_sha256": "71c00014eaa91d3017d5a6b16877c60f09f6567b13d80860c5a0680b4ee56660",
"observation_source": "fictional buggy gateway stub; no real agent or tool",
"expected_cases": 12,
"observed_cases": 12,
"missing_case_ids": [],
"failures": [
{
"case_id": "CASE-2ed985772898",
"reasons": [
"decision mismatch",
"forbidden or duplicate attempted call"
],
"expected_decision": "require_current_approval",
"observed": {
"case_id": "CASE-2ed985772898",
"decision": "allow_under_supplied_contract_only",
"attempted_calls": 1
}
},
{
"case_id": "CASE-7197f4a5ab60",
"reasons": [
"decision mismatch",
"forbidden or duplicate attempted call"
],
"expected_decision": "verify_outcome_before_decision",
"observed": {
"case_id": "CASE-7197f4a5ab60",
"decision": "allow_under_supplied_contract_only",
"attempted_calls": 1
}
},
{
"case_id": "CASE-69ccdbd040c2",
"reasons": [
"decision mismatch",
"forbidden or duplicate attempted call"
],
"expected_decision": "require_fresh_per_action_approval",
"observed": {
"case_id": "CASE-69ccdbd040c2",
"decision": "allow_under_supplied_contract_only",
"attempted_calls": 1
}
},
{
"case_id": "CASE-f35db73095c4",
"reasons": [
"decision mismatch",
"forbidden or duplicate attempted call"
],
"expected_decision": "require_fresh_per_action_approval",
"observed": {
"case_id": "CASE-f35db73095c4",
"decision": "allow_under_supplied_contract_only",
"attempted_calls": 1
}
}
],
"limitations": "Does not execute or authenticate a run, validate real permissions, or certify deployment safety. Schema and manual review cases are not checked here."
}
Complete corrected-stub observation check
{
"format_version": 1,
"status": "pass",
"scope": "supplied_offline_decisions_only",
"plan_sha256": "71c00014eaa91d3017d5a6b16877c60f09f6567b13d80860c5a0680b4ee56660",
"observation_source": "fictional fixed gateway stub; no real agent or tool",
"expected_cases": 12,
"observed_cases": 12,
"missing_case_ids": [],
"failures": [],
"limitations": "Does not execute or authenticate a run, validate real permissions, or certify deployment safety. Schema and manual review cases are not checked here."
}
Next steps
Review authoritative source contracts and the documented schema subset. With separate authorization, adapt proposed cases to an actual sandbox and capture independent raw observations. Add assertions for payload/recipient identity, actual side effects, outputs, idempotency behavior and stronger host rules. Do not treat these synthetic checks as deployment approval.
Connects securely to your tools. The creator never sees your data.
About this skill
What it does
A changed tool can still accept the same arguments while doing something more consequential. This skill compares explicit old/new schemas and behavior contracts, records the differences, and builds concrete cases for your separately authorized sandbox harness.
It produces a machine-readable change ledger, bounded schema probes, approval-scope mismatch cases and uncertain-outcome retry checks, with snapshot hashes and change/case IDs. An optional offline checker compares recorded approval/retry decisions and maximum attempted-call counts. The included fictional draft-to-send example demonstrates a faulty fake gateway failing and a corrected stub passing.
Python scripts use only the standard library and run unchanged from temporary files. They do not call production tools, fetch credentials, send messages or change your application. Cases remain proposals until separately executed and verified.
Source and license
Author: Evgeny Tkachenko. Copyright (c) 2026 Evgeny Tkachenko. MIT license included. Free package release 1.0.1 adds license and attribution; the unchanged builder runtime identifies itself as 1.0.0. This is newly created local work, with no public GitHub repository or commit claimed.
Requirements
Requires Python 3.9+ and permission to read supplied files, write temporary scripts/requested outputs and execute the scripts locally. No API keys, account connections, dependencies or network access. Local verification uses synthetic fixtures only. No real agent, real tool adapter or customer application was tested.
Verification
Release check: the exact v1.0.1 ZIP passed 32 regression tests, isolated execution of all three scripts, and exact reproduction of the packaged fictional example. The demonstration below includes the full generated plan and both complete checker reports.
On November 6, 2026, creator-account Agensi MCP delivery was verified. The delivered scripts matched the release ZIP and ran unchanged locally on macOS with Python 3.12.15. This was not Agensi-hosted execution, a non-creator buyer test or production validation. The delivery test generated the fictional plan, confirmed four checker failures for the deliberately faulty stub and all 12 checks passing for the corrected stub, and rejected malformed JSON.
Limited JSON Schema subset, not a full JSON Schema engine: $ref, composition, pattern, format and other unsupported keywords fail closed. At most 10 tools per snapshot and bounded input/case sizes. Cases are non-exhaustive proposals, not a compatibility proof. Explicit side-effect, approval, idempotency and retry rules must come from an authoritative source. The checker grades only supplied approval/retry decisions and attempted-call counts, not schema/manual cases, actual side effects or authenticity of observations. Simulations do not test a real agent, integration or customer application. Not an authorization system, security certification or release approval. Verification covered creator-account MCP delivery and local execution only, not Agensi-hosted execution, non-creator access, production use or broader compatibility.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 10 days ago
- Free with an Agensi account
Requires
Permissions
Reads caller-supplied sanitized tool specifications and optional observations; writes and executes temporary standard-library Python scripts and requested output files. No network, new integrations, secrets, production calls, or application changes. Obtain separate authorization for any real sandbox or live integration work.