prompt evolve

    1

    A data-driven prompt optimization loop that scores, versions, and evolves prompts against test sets.

    Free

    1 useSecurity scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    prompt evolve

    Example session with this skill installed

    Evolve the prompt in ./agent-eval against testset.jsonl, grading each output against its rubric. Stop after 3 generations or once the average score reaches 0.95.

    • Read your context and instructions
    • Compiled the prompt evolve

    How the loop runs in ./agent-eval (a walkthrough of the steps, not a recorded run):

    1. Render prompt.tmpl once per case in testset.jsonl and run each rendered prompt.
    2. Grade each output 0–1 against its case's rubric and write one {"input": ..., "output": ..., "score": ..., "note": ...} line per case to outputs.jsonl, in test-set order.
    3. Score and save the generation:
      python3 pevolve.py score --dir ./agent-eval --version 1 --outputs outputs.jsonl --scorer rubric --save-prompt prompt.tmpl
    4. Rewrite the prompt from the failing case ids, bump the version, and repeat until 3 generations or an average score of 0.95.

    What you end up with

    • Every prompt version kept as generations/v1.tmpl, v2.tmpl, …
    • generations/scores.jsonl with per-case scores and the failing case ids
    • report.md with the score curve and the best version so far

    Next steps

    • Diff the best generation against v1.tmpl to see what changed.
    • Run python3 pevolve.py report --dir ./agent-eval to regenerate the report.
    • Commit generations/ so every prompt change is reviewable.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Stop guessing whether a prompt change actually worked. This skill replaces trial-and-error manual editing with a measured optimization loop that scores outputs against a test set, identifies specific failures, and evolves the prompt version-by-version. It treats prompt engineering like software engineering, persisting every iteration as a diffable file.

    What it does

    • Systematic optimization runs prompts against a defined test set to measure real performance gains.
    • Version tracking saves every iteration in a generations/ directory for diffing and rollbacks.
    • Failure analysis identifies specific test cases where the prompt fails to generate actionable improvement data.
    • Two ways to score: deterministic checks (exact match, regex, required JSON keys), or your agent grades each output 0–1 against the case's rubric and --scorer rubric saves those grades.
    • Progress reporting generates a report.md tracking the score curve and highlighting the best-performing version.

    How it works

    1. Initialize the workspace with a prompt.tmpl containing an {input} placeholder and a testset.jsonl file.
    2. Execute iterations where you render the prompt, capture outputs, and score them with a deterministic scorer or by grading each output against its rubric.
    3. Persist results using the pevolve.py score command to save the current version and its performance metrics.
    4. Refine the prompt based on reported failures and repeat until the target score is met or the loop plateaus.

    Frameworks & tools

    The skill ships a Python 3 CLI (pevolve.py, standard library only) for file management and scoring. The CLI never calls a model: your agent runs the prompt (and grades the outputs when you score by rubric), and pevolve.py scores or saves those grades. Written for Claude Code.

    Why this beats prompting it yourself

    Manual tweaking is subjective and prone to regression where fixing one case breaks three others. This skill enforces a data-driven approach that ensures the "best" prompt is chosen based on evidence, not intuition, while maintaining a full audit trail of every change.

    Use cases

    • Tuning an extraction prompt until every output contains the required JSON keys.
    • Tuning a support-reply or writing prompt against a quality rubric.
    • Debugging edge-case failures in a complex classification task.

    Known limitations

    The prompt template only supports a literal {input} placeholder. Deterministic JSON schema scoring is limited to checking for the presence of specific keys rather than full validation. Rubric grades come from your agent, so they are only as consistent as its judgement; keep rubrics specific.

    Templates only support a single {input} placeholder. JSON schema scorer only checks key existence. CLI does not make network calls; LLM scoring must be handled by the agent.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    1 use

    Times used to date

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 25 days ago

    • Free with an Agensi account

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Network Access
    Environment Variables
    Listed28 days ago
    Updated25 days ago

    What's inside

    Frequently Asked Questions

    You score every prompt version against the same test set, see exactly which cases fail, and keep every version on disk, so a change that breaks other cases shows up in the scores.

    It is written for Claude Code. It needs Python 3 to run the bundled `pevolve.py` CLI (standard library only, no extra packages).

    Four scorers: exact match, regex, a required-JSON-keys check, and `rubric`, which saves your agent's own 0–1 grade (and optional note) for each case.

    Three files: `SKILL.md` (the workflow), `scripts/pevolve.py` (the CLI) and `LICENSE`. You write your own `prompt.tmpl` and `testset.jsonl`.

    Currently, the template system is optimized for a single `{input}` placeholder; more complex multi-variable templates may require pre-processing before being passed to the optimization loop.

    Each scored version is saved as `generations/vN.tmpl` with its scores in `generations/scores.jsonl`; `pevolve.py report` writes `report.md` with the score curve and the best version.

    New in Prompt & Skill Engineering