- Home
- Skills
- Prompt & Skill Engineering
- prompt evolve
prompt evolve
A data-driven prompt optimization loop that scores, versions, and evolves prompts against test sets.
Free
Works with the AI Agents you already use
prompt evolve
Example session with this skill installed
Evolve the prompt in ./agent-eval against testset.jsonl, grading each output against its rubric. Stop after 3 generations or once the average score reaches 0.95.
- Read your context and instructions
- Compiled the prompt evolve
How the loop runs in ./agent-eval (a walkthrough of the steps, not a recorded run):
- Render
prompt.tmplonce per case intestset.jsonland run each rendered prompt. - Grade each output 0–1 against its case's
rubricand write one{"input": ..., "output": ..., "score": ..., "note": ...}line per case tooutputs.jsonl, in test-set order. - Score and save the generation:
python3 pevolve.py score --dir ./agent-eval --version 1 --outputs outputs.jsonl --scorer rubric --save-prompt prompt.tmpl - Rewrite the prompt from the failing case ids, bump the version, and repeat until 3 generations or an average score of 0.95.
What you end up with
- Every prompt version kept as
generations/v1.tmpl,v2.tmpl, … generations/scores.jsonlwith per-case scores and the failing case idsreport.mdwith the score curve and the best version so far
Next steps
- Diff the best generation against
v1.tmplto see what changed. - Run
python3 pevolve.py report --dir ./agent-evalto regenerate the report. - Commit
generations/so every prompt change is reviewable.
Connects securely to your tools. The creator never sees your data.
About this skill
Stop guessing whether a prompt change actually worked. This skill replaces trial-and-error manual editing with a measured optimization loop that scores outputs against a test set, identifies specific failures, and evolves the prompt version-by-version. It treats prompt engineering like software engineering, persisting every iteration as a diffable file.
What it does
- Systematic optimization runs prompts against a defined test set to measure real performance gains.
- Version tracking saves every iteration in a
generations/directory for diffing and rollbacks. - Failure analysis identifies specific test cases where the prompt fails to generate actionable improvement data.
- Two ways to score: deterministic checks (exact match, regex, required JSON keys), or your agent grades each output 0–1 against the case's rubric and
--scorer rubricsaves those grades. - Progress reporting generates a
report.mdtracking the score curve and highlighting the best-performing version.
How it works
- Initialize the workspace with a
prompt.tmplcontaining an{input}placeholder and atestset.jsonlfile. - Execute iterations where you render the prompt, capture outputs, and score them with a deterministic scorer or by grading each output against its rubric.
- Persist results using the
pevolve.py scorecommand to save the current version and its performance metrics. - Refine the prompt based on reported failures and repeat until the target score is met or the loop plateaus.
Frameworks & tools
The skill ships a Python 3 CLI (pevolve.py, standard library only) for file management and scoring. The CLI never calls a model: your agent runs the prompt (and grades the outputs when you score by rubric), and pevolve.py scores or saves those grades. Written for Claude Code.
Why this beats prompting it yourself
Manual tweaking is subjective and prone to regression where fixing one case breaks three others. This skill enforces a data-driven approach that ensures the "best" prompt is chosen based on evidence, not intuition, while maintaining a full audit trail of every change.
Use cases
- Tuning an extraction prompt until every output contains the required JSON keys.
- Tuning a support-reply or writing prompt against a quality rubric.
- Debugging edge-case failures in a complex classification task.
Known limitations
The prompt template only supports a literal {input} placeholder. Deterministic JSON schema scoring is limited to checking for the presence of specific keys rather than full validation. Rubric grades come from your agent, so they are only as consistent as its judgement; keep rubrics specific.
Templates only support a single {input} placeholder. JSON schema scorer only checks key existence. CLI does not make network calls; LLM scoring must be handled by the agent.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
1 use
Times used to date
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 25 days ago
- Free with an Agensi account