- Home
- Skills
- Code Quality & Review
- Rule Calibrate
Works with the AI Agents you already use
Rule Calibrate
Dry-run your CLAUDE.md rules against your recent commits and see which are noisy, vague or never fire, before you enforce them.
Free
Rule Calibrate
Example session with this skill installed
Run rule-calibrate on this repo. How would my CLAUDE.md rules have done against the last 15 commits?
- Read your context and instructions
- Compiled the rule calibrate
Summary
One rule is
noisy (it fires on everything), two are
decisive (they catch real violations), and one is
quiet (it never fired). 10 of 10 changes were judged against 4 rules.
This is real output from the packaged skill on a small throwaway repo (notes-app, 11 commits, 4 rules in CLAUDE.md). For this run I pointed the skill at a tiny local stand-in that marks a rule as firing when its backticked word appears in the change. It shows the flow and the table format. It says nothing about how a real model judges your rules.
The result
10 of 10 hunks judged against 4 rules (fire level 0.8)
10/10 median 0.95 max 0.95 noisy Always keep `import` lines sorted and at the top of the file.
4/10 median 0.05 max 0.95 decisive Never leave `print(` calls in library code, use logging.
2/10 median 0.05 max 0.95 decisive Never use `eval` anywhere in the codebase.
0/10 median 0.05 max 0.05 quiet Always add type hints to public functions.
How to read each line
| Rule | Fired | Verdict | What to do |
|---|---|---|---|
Keep import lines sorted | 10 of 10 | noisy | It fires on every change, so enforcing it would block everything. Reword or narrow it first. |
No print( in library code | 4 of 10 | decisive | Separates violations from clean changes. Safe to enforce. |
Never use eval | 2 of 10 | decisive | Same. |
| Type hints on public functions | 0 of 10 | quiet | The code follows it, or the rule is too vague to notice. Make it concrete if you want it enforced. |
Next steps
- Reword the noisy rule, then run it again.
- For each quiet rule, check one commit by hand: does the code really follow it?
- When the table looks sane, turn rule enforcement on in the jev-tools plugin.
- Use
--commits 30 --hunks 60for a longer look, at more tokens.
Connects securely to your tools. The creator never sees your data.
About this skill
Hey! Before you let any tool block your edits for breaking a project rule, find out which of your rules are actually any good.
This skill is a dry run for your rulebook. It takes the "never / always / must" bullets from your CLAUDE.md (or AGENTS.md), replays your recent commits against each one, and tells you how often every rule would have fired.
What you get
One line per rule: fired/hunks median max verdict rule, with a verdict:
- noisy: fires on 60% or more of real changes. Reword or narrow it before you enforce it.
- weak: never decisive. Make the rule concrete.
- quiet: never came close to firing. Either the code follows it, or the rule is too vague to notice.
- decisive: separates violations from clean changes.
- skipped: fewer than 5 changes could be judged.
What it reads
- Rules: imperative bullets ("never", "always", "must", "do not", "avoid") from
CLAUDE.md,AGENTS.mdand.claude/rules/*.mdin the folder you run it from. Up to 20 rules. Prose paragraphs are ignored, and~/.claude/CLAUDE.mdis never read. - Changes: the diffs of your last N non-merge commits, code files only (docs, JSON, lock files and images are skipped, as are secret-looking file names), each capped at 8,000 characters.
Your code leaves your machine
This one needs a model, and every judged change is sent to it. That is up to 8,000 characters of a real commit diff each, to api.codiv.ai (free Codiv OpenJev key in OPENJEV_API_KEY) or to a loopback address for a local model set up with jev-tools-setup. Secret-shaped strings are redacted first and the key is only ever sent to those hosts. At 20 rules, expect about 5,000 input tokens per change, so 30 changes is roughly 150,000 tokens. Don't run it on private history you aren't willing to send.
What you need
Python 3.10+ and git. Standard library only, nothing to install. It cannot run offline: with no key and no local model it exits with a message.
Honest limits
- The evidence is thin. The author replayed 20 rules over 24 real changes from one other project, and no rule fired above 0.35, so every rule read "quiet". That is what a good rulebook on clean history looks like, and also what a blind rulebook looks like. Zero fires is not proof of a good rulebook.
- Shallow clones have one commit. Run
git fetch --deepen=40first. - It appends one small line per model call (timestamp, number of questions, milliseconds, tokens, or an error type) to
~/.jev-tools/log.jsonl, or to the path inJEV_LOG. Diff content and keys are never logged.
Part of jev-tools
One of several skills from the free, MIT-licensed jev-tools project. The full plugin adds the optional rule-enforcement hook this skill helps you tune, which is not included in this listing: github.com/Rcidshacker/jev-tools
Needs Python 3.10+, git and a model (free Codiv OpenJev key or a small local model). Cannot run offline. Reads at most 20 rules (imperative bullets from CLAUDE.md, AGENTS.md or .claude/rules in the current folder) and judges code changes only. Needs at least 5 judged changes per rule for a verdict. Shallow clones have one commit, so run git fetch --deepen=40 first. Only lightly tested; zero fires is not proof of a good rulebook. Tested by the author on Windows only.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
1 use
Times used to date
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 1 month ago
- Free with an Agensi account
Requires
Permissions
Allowed Hosts
File Scopes
Shell runs the bundled Python script and git log. Files: it reads CLAUDE.md, AGENTS.md and .claude/rules in the current folder, and the optional config and credentials files that jev-tools-setup writes in ~/.jev-tools. It appends one small log line per model call there (never diff content or keys). Environment: reads OPENJEV_API_KEY (and JEV_THRESHOLD, JEV_MODE, JEV_LOG) if you set them. Network: it posts commit diffs (up to 8,000 characters each, secret-shaped strings redacted first) to api.codiv.ai, or to a loopback address for a local model; redirects are refused. It cannot run offline.