Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Hardware Model Sizing

    1

    Predict exact LLM fitment for local hardware by calculating weight, KV cache, and VRAM overhead.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Hardware Model Sizing

    Hardware Model Sizing

    Example session with this skill installed

    I have a machine with two GPUs, a 3080 10GB and a 4090 24GB. I need to run a 32B model at 32k context for coding. What fits?

    • Read your context and instructions
    • Compiled the hardware model sizing

    I have inventoried your hardware and calculated the usable memory pools. Because these are discrete GPUs, they must be treated as separate pools rather than a 34GB sum. ## Hardware Sizing Audit **Inventory & Usable Budget** - **Pool 1:** NVIDIA RTX 4090 (24GB VRAM) -> **22.0 GB usable** - **Pool 2:** NVIDIA RTX 3080 (10GB VRAM) -> **8.0 GB usable** - **Target Context:** 32,768 tokens (FP16 KV) - **KV Cache Overhead:** ~8.2 GB (for 32B class dense models) **Model Recommendations (32B Class)** | Model / Quant | Total Runtime (Weights + KV) | Status (Pool 1) | Status (Pool 2) | | :--- | :--- | :--- | :--- | | 32B Q4_K_M | ~26.2 GB (18.0 + 8.2) | **Offload** (Needs ~4GB CPU) | **Don't Bother** | | 32B Q3_K_L | ~22.1 GB (13.9 + 8.2) | **Fits** (0.1GB slack) | **Don't Bother** | | 32B Q2_K | ~18.8 GB (10.6 + 8.2) | **Fits** (3.2GB slack) | **Don't Bother** | **Analysis** - **Workhorse:** At 32k context, a 32B model is too heavy for the 3080. On the 4090, a Q3_K_L is the highest quality that fits entirely in VRAM. - **Quality Ceiling:** You can run Q4_K_M on the 4090, but you will see a performance drop as ~4GB of layers spill into system RAM. - **Don't Bother:** Avoid running dense 32B models on the 10GB 3080 for this context length; even at 2-bit quants, the KV cache alone consumes nearly the entire pool. **Next steps** - Download the **Q3_K_L** GGUF for the 4090 to maintain full VRAM speed at 32k context. - If quality is prioritized over speed, use the **Q4_K_M** with partial CPU offloading. - Consider a 14B model family if you need high-speed inference on the 3080.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Developers often struggle to predict if a local LLM will actually run on their hardware without hitting Out-of-Memory (OOM) errors. Marketing nicknames and raw file sizes on Hugging Face are misleading because they ignore KV cache overhead, operating system requirements, and the physics of discrete vs. unified memory pools.

    What it does

    • Inventories local hardware to identify discrete VRAM pools and unified memory availability without relying on marketing stickers.
    • Calculates real-world usable budget by subtracting OS and runtime overhead from raw hardware specs.
    • Queries Hugging Face GGUF file trees to sum multi-file shards and verify actual weight sizes on disk.
    • Factors in KV cache memory requirements based on your specific target context length and data type.
    • Categorizes models into "fits", "offload", or "don't bother" buckets for your specific machine.

    Why this beats prompting it yourself

    General-purpose LLMs frequently hallucinate that you can sum the VRAM of two different GPUs or forget that MoE models require the full weight set in memory. This skill uses a deterministic calculation process and live API lookups to ensure you don't waste hours downloading a 40GB model that will OOM at 8k context.

    Use cases

    • Determining the largest model family and quantization level that fits a specific developer workstation.
    • Validating vendor claims about running massive models on entry-level hardware before purchasing.
    • Sizing a machine's model runway to see if it can handle 128k context tasks versus 8k chat.
    • Auditing a multi-GPU setup to decide between tensor splitting or running separate instances.

    Known limitations

    Does not support cloud API model routing, image/video diffusion models, or sizing for training and LoRA fine-tuning. Assumes GGUF/llama.cpp runtime.

    Inference only, no training/LoRA sizing. Limited to GGUF format and llama.cpp runtimes. Requires Python 3.10+ and internet access for Hugging Face API queries.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 2 months ago

    Requires

    Hugging Face
    Python

    Permissions

    Terminal / Shell
    Network Access

    Allowed Hosts

    huggingface.co
    Listed2 months ago

    What's inside

    Frequently Asked Questions

    This skill performs a deterministic analysis of your specific GPU/VRAM architecture and context length requirements to predict if a model will fit, saving you from wasting hours downloading large files that lead to OOM errors.

    The skill currently focuses on the GGUF format and the llama.cpp runtime, as these are the standards for local deployment and multi-file sharding.

    Yes, the skill is designed to detect multiple discrete GPUs and calculate how weights can be split across them, versus how unified memory (like Apple Silicon) handles the load.

    It calculates the exact memory footprint of the KV cache based on your target context window (e.g., 8k vs 128k) and data type, which often determines whether a model fits or crashes during long conversations.

    You will receive the AI agent skill package containing the logic for hardware inventorying, Hugging Face API integration for weight calculation, and the deterministic sizing engine.

    While the skill identifies hardware specs, it currently focuses on inference fitment; it does not calculate the additional memory buffers required for training, gradients, or LoRA fine-tuning.

    New in Agents & Orchestration