- Home
- Skills
- Sales & Lead Gen
- lead score calibration analyst
Works with the AI Agents you already use
lead score calibration analyst
This skill evaluates existing fit-plus-engagement lead scores against historical conversion outcomes using a deterministic execution environment.
$2.99
lead score calibration analyst
Example session with this skill installed
Check if our Q3 lead score is calibrated. We use 60% fit and 40% engagement. I have the lead outcomes from the last 90 days across tech and retail sectors. Use the attached leads.csv.
- Read your context and instructions
- Compiled the lead score calibration
- Generated the data export
I have processed the Q3 lead data against your 60/40 weighting model. The analysis was restricted to 1,240 mature, consented records.
Lead Score Calibration — Acme Corp, 2023-10-27
1. Scope, sources and assumptions
- Sources: E1 (leads.csv)
- Approver: Operations Manager (Scope: Sales Analytics)
- Weights: Fit 0.60, Engagement 0.40
- Window: 90 days
2. Cohort and exclusions
| Reason | Count |
|---|---|
| Total Records | 1,500 |
| Lacks Consent | 150 |
| Outside Scope | 60 |
| Immature (Open) | 50 |
| Analyzed | 1,240 |
3. Band calibration
| Band | Min score | Leads | Conversions | Observed | 95% low | 95% high | Lift | Stated | Gap | Check |
|---|---|---|---|---|---|---|---|---|---|---|
| Hot | 80 | 200 | 40 | 20.0% | 14.5% | 25.5% | 2.5x | 25.0% | -5.0% | Review |
| Warm | 50 | 540 | 54 | 10.0% | 7.5% | 12.5% | 1.2x | 12.0% | -2.0% | Pass |
| Cold | 0 | 500 | 5 | 1.0% | 0.1% | 1.9% | 0.1x | 2.0% | -1.0% | Pass |
4. Discrimination and weight search
- Combined AUC: 0.72 (Good separation)
- Fit AUC: 0.68
- Engagement AUC: 0.61
- Weight Search: A weight of 0.70 Fit / 0.30 Engagement showed a marginal AUC gain of 0.03 in-sample.
5. Subgroup calibration
| Value | Leads | Conversions | Observed | Expected | O/E | Status |
|---|---|---|---|---|---|---|
| Tech | 800 | 80 | 10.0% | 9.8% | 1.02 | Stable |
| Retail | 440 | 19 | 4.3% | 7.5% | 0.57 | Underperforming |
6. Proposed actions (drafts, approval required)
| ID | Type | Description |
|---|---|---|
| PA-01 | Rate Adjustment | Lower 'Hot' stated rate to 20% (Management decision required) |
| PA-02 | Segment Review | Investigate Retail sector score drift (Management decision required) |
7. Warnings and items for human review
- Warning: Small group suppression applied to "Education" sector (n<5).
- Computed result: Observed conversion for 'Hot' leads is significantly below the stated 25% target.
Analytical estimate — validation required: The Retail segment shows poor calibration; the fit model may not account for retail-specific procurement cycles.
This analysis supports, and does not replace, professional judgment and management's decision.
Next steps
- Review the 'Hot' band conversion gap to determine if marketing expectations need resetting.
- Approve the investigation into the Retail segment underperformance.
- Validate the proposed 70/30 weight shift using a hold-out dataset.
leads.csv
CSV · data export
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
Stop guessing if your lead scoring model works. This skill evaluates existing fit-plus-engagement lead scores against historical conversion outcomes using a deterministic execution environment. It identifies whether your high-priority bands actually convert at higher rates, if your stated conversion targets match reality, and if the model remains predictive across different business segments.
What it does
- Calibration testing measures observed vs. expected conversion rates per score band with 95% confidence intervals.
- Rank discrimination calculates AUC (Area Under Curve) for combined, fit, and engagement scores to see which truly drives conversion.
- Subgroup analysis detects performance drift across industries, channels, or regions while suppressing small groups for privacy.
- Weight optimization runs a grid search to propose potential re-weighting of fit vs. engagement as a draft for review.
- Data hygiene enforcement filters records based on consent, access scope, and provenance before running any calculations.
How it works
- Prepare request: The agent maps your model weights, score bands, and lead outcome data into a structured schema.
- Execute script: A deterministic Python script processes the data to ensure mathematical accuracy without LLM hallucination.
- Analyze envelope: The agent interprets the script's output status, including warnings for band inversions or low AUC.
- Generate report: You receive a structured audit including calibration tables, discrimination metrics, and draft proposals.
Frameworks & tools
This skill uses a self-contained Python execution environment and deterministic scripts to handle statistical calculations. It integrates with CRM data exports provided in JSON format.
Why this beats prompting it yourself
Generic prompts often hallucinate statistical significance or fail to handle the math of confidence intervals and AUC correctly. This skill uses a hard-coded script to ensure every calculation is reproducible and follows strict privacy protocols like small-group suppression.
Use cases
- Audit an existing lead scoring model to see if "A" leads actually convert better than "D" leads.
- Validate if a marketing team's "expected conversion rates" for score bands are statistically realistic.
- Identify if a lead score is biased or underperforming in specific industries or sales channels.
- Test if changing the weight of engagement vs. fit would have historically improved rank discrimination.
Known limitations
Does not design new scoring rules from scratch. Rejects analysis by protected personal attributes. Calculations are in-sample only and require historical outcome data to function.
Requires historical conversion outcomes. Rejects protected personal attributes (age, gender, etc.). Weight search is in-sample only. Not for designing new scoring rules.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 2 days ago