Works with the AI Agents you already use
Local LLM Fit and Benchmark Kit
Check whether a local LLM fits your GPU and how fast it runs. Three standard library Python scripts estimate memory, benchmark Ollama or vLLM, and write a scored report that ranks your models.
$5
Local LLM Fit and Benchmark Kit
Example session with this skill installed
I have a 24 GB GPU and Ollama is running on this machine. Check whether llama3.1:8b, qwen2.5:14b and qwen2.5:32b fit with an 8k context, benchmark the ones that do, and tell me which is best for a chat assistant.
- Read your context and instructions
- Compiled the local llm fit
Summary
All three models fit on a 24 GiB GPU at 8,192 tokens of context. The 32B model is tight. [Replace with one sentence on which model scored best and why, from your run.]
Memory check (GiB, 8,192 token context)
| Model | Quant | Weights | KV cache | Total | Share of GPU | Verdict | Longest context that fits |
|---|---|---|---|---|---|---|---|
| llama-3.1-8b | q4_k_m | 4.53 | 1.00 | 6.53 | 27% | comfortable | about 151,000 |
| qwen2.5-14b | q4_k_m | 8.30 | 1.50 | 10.80 | 45% | comfortable | about 80,000 |
| qwen2.5-14b | q8_0 | 14.55 | 1.50 | 17.05 | 71% | comfortable | about 46,000 |
| qwen2.5-32b | q4_k_m | 18.35 | 2.00 | 21.35 | 89% | tight | about 19,000 |
For comparison, llama-3.1-70b at q4_k_m needs 43.36 GiB (181% of this GPU) and does not fit.
Benchmark (Ollama on [your GPU name], 5 runs, 128 tokens each)
| Model | Decode tokens per second | Time to first token (s) | Success | Score | Verdict |
|---|---|---|---|---|---|
| llama3.1:8b | [from your run] | [from your run] | [x/5] | [score] | [verdict] |
| qwen2.5:14b | [from your run] | [from your run] | [x/5] | [score] | [verdict] |
| qwen2.5:32b | [from your run] | [from your run] | [x/5] | [score] | [verdict] |
Recommendation
[Replace with your agent's actual recommendation from the report.]
Next steps
- Re-run with
--concurrency 4if more than one person will use the server. - A tight verdict leaves little room, so shorten the context or use a smaller quantization before relying on it.
- Run
--listto see the built in presets, or pass your own model's numbers from its model card.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
You pick a model and a quantization, then guess whether it fits your GPU and quote speeds nobody measured on your hardware. A chat model asked to guess will sound sure and can be wrong.
What you get
- vram_fit.py works out weights, KV cache and overhead, says comfortable, tight or does not fit, and gives the longest context that fits.
- bench_endpoint.py streams from your Ollama or OpenAI compatible server (such as vLLM) and records time to first token, tokens per second, failed requests and total throughput with several users at once.
- make_report.py builds a Markdown report that ranks your models with a 0 to 100 readiness score and a plain verdict for each.
- SKILL.md tells your agent when to run each script and how to explain the results.
Why this beats prompting it yourself
A model cannot measure your hardware. These scripts time real requests, count failures, and use the same formula every run, so two models are compared fairly.
Known limitations
- You give the GPU memory or paste nvidia-smi text. It does not detect the GPU.
- Six model presets are built in and they are approximate. Any other model works if you enter its numbers from the model card.
- Multi GPU splits, CPU offload and speculative decoding are not modelled.
- It checks fit and speed. It does not test answer quality.
- Tested with a mock streaming server and hand computed memory cases. Results on your machine depend on your setup.
No support for multi-GPU, CPU offload, or speculative decoding. Architecture presets are approximate; exact model card parameters preferred for custom models.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
Requires