More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Fine-Tune a Small Model, Then Prove It

    1

    Teach a small model your yes/no and risk-score decisions from your own past calls — then make it beat the untuned model on data it never saw, or it doesn't ship.

    $15

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Fine-Tune a Small Model, Then Prove It

    Example session with this skill installed

    Fine-tune our small decision model on this labeled dataset, then evaluate it against the untuned baseline and tell me whether we should serve it.

    • Read your context and instructions
    • Compiled the fine-tune a small

    Evaluation complete — held-out set, 200 labeled questions.

    Baseline (untuned): accuracy 0.71, ECE 0.14, Brier 0.21
    Fine-tuned (LoRA): accuracy 0.83, ECE 0.06, Brier 0.13

    Accuracy gain: +0.12. Paired bootstrap 95% confidence interval on the gain: +0.07 to +0.17 — the gain is real, not noise.

    Per-question deltas: 34 questions fixed, 10 regressed. The regressions are listed with their questions and labels, so you can see exactly what got worse before you decide.

    Calibration: expected calibration error fell from 0.14 to 0.06 and Brier score from 0.21 to 0.13, so the tuned model's probabilities are more trustworthy, not just more often right.

    Gate decision: SERVE. Accuracy, calibration, and the bootstrap interval all pass the eval-gate checklist, with regressions reviewed and acceptable. If any gate had failed, the decision would have been DO NOT SERVE, with the failing metric named.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    A decision model is not a chatbot. It takes a situation plus typed questions — allow or block, which team owns this ticket, how risky is this on a scale of 1 to 5 — and returns a probability for each answer. Fine-tuning one on your labeled examples is the easy half, and it's the half most people stop at. This skill's real subject is the proof. A tuned model is a claim, and the evaluation is where the claim gets tested.

    Inside the package: the skill file covering the dataset format, the split and leakage checks, the training recipe, the metrics, and the gate; an evaluation-gate checklist; and an example dataset file with deliberately tricky guardrail cases. The dataset is one JSON object per line: the situation, the questions (yes/no, pick-one, or a numbered scale), and your answers. Training uses LoRA — a method that trains a small add-on instead of the whole model — starting from a known-good recipe for the model family. A single 24 GB graphics card handles a roughly 9B model; 50 to 200 labeled examples per question is a reasonable start, drawn from real past decisions wherever possible, because invented examples teach the model your imagination instead of your operation.

    A run goes like this: hold out about a fifth of the examples as a test set with a fixed split, and check it for near-duplicates of training examples first — a test item that almost repeats a training item measures memory, not learning. Train, then run the tuned model and the untuned baseline on the same test set and read all of it: accuracy overall and per question; calibration — when the model says 80%, is it right about 80% of the time; a score that punishes confident wrong answers; the list of individual questions that got fixed and the list that got worse, read by name; and a statistical resampling check that the improvement is real and not luck on a small test set. The gate is hard and set before training: accuracy clears your floor, the improvement is statistically real, calibration did not get worse, the probability score improved or held, and you've read and accepted every regression. Fail any one and the answer is more labels for the weak spots and a retrain — not deployment.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 6 days ago

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Listed6 days ago

    What's inside

    Frequently Asked Questions

    Teach a small model your yes/no and risk-score decisions from your own past calls — then make it beat the untuned model on data it never saw, or it doesn't ship.

    This skill uses the open SKILL.md standard. Use it through an MCP-compatible agent connected to Agensi, and check the compatibility information on this page for confirmed agents and required capabilities.

    Connect your AI agent to Agensi through MCP, then ask it to use Fine-Tune a Small Model, Then Prove It on your task. Your agent loads the latest approved version each time. Paid skills require a purchase or active subscription. Skill files are not available for download; your licence does not allow you to save, copy or export any part of a skill.

    New in LLM Ops