More screenshots
Works with the AI Agents you already use
TRL Fine-Tuning: SFT, DPO and RLHF Workflows
Pick the right post-training method (SFT, DPO, GRPO, PPO), gate your data before training, and ship on held-out behavior, with recipes, heuristics, and a readiness rubric.
$5
TRL Fine-Tuning: SFT, DPO and RLHF Workflows
Example session with this skill installed
We have a 7B instruct model, about 40k logged support chats, and thumbs up/down from users on the bot's answers. We want it to answer in our tone, stay in scope, and stop inventing policy details. One 24GB GPU. Plan the post-training.
- Read your context and instructions
- Compiled the trl fine-tuning
Method pick, from what you have: two stages. Stage 1 is SFT — the thumbs logs tell you which answers users liked, but they are not clean demonstrations, so first build 3–5k prompt→response pairs from your best resolved chats, rewritten to the tone and scope rules you want. Stage 2 is DPO, using the thumbs data as chosen/rejected pairs. PPO is ruled out on hardware alone (four models will not fit one 24GB GPU) and GRPO is ruled out because "sounds like our tone" is not a verifiable reward.
Data gates, applied before any training: deduplicate the chat logs by intent (support logs repeat the same 200 questions thousands of times — that teaches one trick, not support). Hand-read 30 thumbs pairs: expect to find up-votes on confident wrong answers; any pair where the chosen answer invents a policy detail gets dropped, and that failure becomes an SFT example teaching the in-scope refusal instead. Length audit: check whether up-voted answers run longer — if they do, balance before DPO or the model learns length, not quality. One format contract across every example.
Baseline first: score the current model on 300 held-out real prompts — in-scope accuracy, invented-policy rate, tone match — plus a small general instruction set to price forgetting later. Config: SFT with LoRA, LR 1e-4, 2 epochs; then DPO from the SFT checkpoint, beta 0.1, one epoch, effective batch 64. Watch the invented-policy rate at every eval, not just win-rate.
Ship bar, set now: invented-policy rate down at least half versus baseline, tone match up, general-set regression inside 2 points. If DPO's win-rate moves but the invention rate does not, the pairs are teaching preference without facts — the fix is more refusal SFT data, not a beta sweep.
Connects securely to your tools. The creator never sees your data.
About this skill
Most post-training projects fail before the first training step: the wrong method for the data on hand, preference pairs nobody ever audited, and a ship decision made by staring at training loss. This skill runs the work in the order that prevents those failures. It opens with an inputs/outputs contract and a stop rule — no pairs and no scoring function means you have a data collection problem, not a training problem. The method picker maps what you actually have (prompt-response pairs, chosen/rejected pairs, a verifiable reward, or nothing yet) to the simplest method that fits, with the standing rule that if DPO can do it, you do not pay PPO's instability and debugging tax. Four data readiness gates come next — prompt diversity, a 30-example label spot-check, a length audit against the classic longer-equals-better reward hack, and a single format contract — and a failed gate means fixing data, not tuning hyperparameters. Each method then gets a minimal working recipe: SFT as the foundation, DPO with beta treated as the whole game, GRPO for verifiable rewards without a critic, PPO with its KL discipline, and reward modeling with a calibration check before anything downstream trusts it. The run protocol baselines the starting model on a held-out set, sets the ship bar before training, and prices catastrophic forgetting on a general instruction set before shipping. A fill-in run-plan template, a five-dimension ship-readiness rubric (ship at 8+/10, no zeros on data gates or held-out evaluation), a hardware reality check from one 24GB GPU to 70B tensor parallelism, a failure-mode table, and a complete GRPO worked example close it out.
- It plans and judges training runs; it does not execute training or ship code for your cluster.
- Hyperparameter starting points are heuristics for common model sizes, not guarantees — your held-out metric is the arbiter.
- It covers post-training of an existing model, not pre-training, data labeling operations, or serving.
- Preference-based methods cannot fix a model whose base capabilities are missing; the skill will tell you to go back to SFT data instead.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 24 days ago
Permissions
No special permissions declared or detected