Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Hybrid Local and Cloud LLM Router with Eval Gate

    1

    Route each LLM call to Ollama, llama.cpp, LM Studio, vLLM or cloud by task, privacy, quality and cost, with code and evals. ROUTE LOCAL / TEST FIRST / KEEP CLOUD

    $7

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Hybrid Local and Cloud LLM Router with Eval Gate

    Hybrid Local and Cloud LLM Router with Eval Gate

    Example session with this skill installed

    Our support-desk app sends three LLM calls to a cloud model: ticket triage (120k/month), customer-field extraction with PII (40k/month), and reply drafting (30k/month). We pay $2.50 per 1M input and $10 per 1M output tokens and have one RTX 4090. Which calls should go to a local Ollama model, and give me the Python router.

    • Read your context and instructions
    • Compiled the hybrid local

    Local/Cloud Routing Plan — support-desk

    Overall verdict: ROUTE LOCAL — ticket triage passed the eval gate; PII extraction must move local for privacy but is TEST FIRST until its eval runs.

    Routing policy

    Call typeVol/moTokens in/outSensitiveTargetVerdict
    triage_ticket120,000900 / 20nolocal, cloud fallbackROUTE LOCAL
    extract_customer_fields40,0001,500 / 150yeslocal onlyTEST FIRST
    draft_reply30,0003,000 / 400nocloudKEEP CLOUD

    Hardware fit (estimates, RTX 4090 24 GB)

    Small class (~8B) at ~4.8 bits: 8 x 4.8 / 8 = 4.8 GB weights. KV at 8,192 ctx x 4 slots ~ 4 GB. (4.8 + 4) x 1.15 ~ 10.1 GB. Fits.

    Eval gate (your 50 labeled tickets)

    CallCloudLocalSchema-validPass
    triage_ticket47/50 (94%)46/50 (92%)50/50yes (within 2 pts)
    extract_customer_fieldsnot runnot run-pending

    Cost model (your prices: $2.50 in / $10 out per 1M)

    • triage: 108M x 2.50 + 2.4M x 10 = $270 + $24 = $294
    • extract: 60M x 2.50 + 6M x 10 = $150 + $60 = $210
    • draft: 90M x 2.50 + 12M x 10 = $225 + $120 = $345
    • Today: $849/mo
    • Routed: draft $345 + triage escalations $294 x 8% (assumed) = $23.52 + extract $0 = $368.52
    • Savings: about $480/mo (estimate; confirm escalation rate in shadow mode)

    Router scaffold (excerpt)

    ROUTES = {
      "triage_ticket": Route("local", LOCAL_MODEL, schema=TRIAGE_SCHEMA),
      "extract_customer_fields": Route("local", LOCAL_MODEL, sensitive=True),
      "draft_reply": Route("cloud", CLOUD_MODEL),
    }
    

    Escalates on timeout, bad JSON after 1 retry, or confidence < 0.7. Sensitive calls never escalate and their prompts are never logged.

    Rollout

    1. Shadow triage for 7 days, compare labels. 2. Switch triage. 3. Run the extraction eval, then cut it over.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Most apps send every LLM call to the same cloud model, including small jobs like labeling a ticket or pulling fields out of a form. That costs money, adds latency, and can send sensitive data off the machine when it does not need to leave.

    This skill works through your app's LLM calls one by one and builds a router that sends each call to a local model (Ollama, llama.cpp server, LM Studio or vLLM) or a cloud model.

    • Finds every LLM call in the repo and records tokens, output shape, latency need and data sensitivity
    • Sorts calls into local candidates (classify, extract, short summaries, triage) and cloud work (multi-step reasoning, long context, critical code changes)
    • Checks hardware fit with the memory math shown and labeled as an estimate
    • Probes your local server's OpenAI-compatible endpoints and structured output support
    • Writes a routing policy table and a Python or TypeScript scaffold: one OpenAI-compatible client, config-driven rules, timeouts, fallback to cloud on failure or low confidence, and structured logs
    • Runs an eval gate so a call only routes local when quality holds on your own test set
    • Builds the cost and latency model from your prices and volumes, step by step
    • Gives each call type, and the plan overall, a verdict of ROUTE LOCAL, TEST FIRST or KEEP CLOUD

    Sensitive data is pinned local by policy and never escalated or logged in plain text.

    For developers and teams running agents or LLM features who want lower API bills or on-device privacy without guessing.

    Honesty note: it does not name a "best" model, it does not invent prices or benchmark scores, and nothing gets ROUTE LOCAL without eval numbers. Third-party results, such as reported cloud-cost cuts from hybrid research, are cited as reported, not promised.

    Memory and savings figures are estimates built from your inputs and must be confirmed with the eval gate and shadow mode. Endpoint features differ by server and release, so the skill probes rather than assumes. It does not recommend a specific model as best, does not fine-tune, and needs web access to re-check current docs.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 1 month ago

    Requires

    Ollama or llama.cpp or LM Studio or vLLM
    Cloud LLM API key

    Permissions

    Terminal / Shell
    Network Access
    Read Files
    Write Files
    Environment Variables
    Listed1 month ago

    What's inside

    Frequently Asked Questions

    A SKILL.md with the step-by-step workflow, verdict rules, output template and checklist, plus references/routing-reference.md with endpoint notes for Ollama, llama.cpp, LM Studio and vLLM, task-class table, memory and cost formulas, a Python router scaffold, a TypeScript scaffold and an eval gate script pattern.

    Any agent that reads SKILL.md skills, such as Claude Code, Codex CLI, Cursor, VS Code Copilot and Gemini CLI. Behavior depends on what tools the agent has, such as shell and web access.

    Unzip the folder into your agent's skills directory and ask about routing your LLM calls. To run probes and evals you need a local server running and the openai package (plus jsonschema for Python) installed in your project.

    Hardware figures are estimates, endpoint support changes between server releases and must be probed, and savings depend on your own eval results and escalation rate. It does not pick or download models for you.

    It follows a fixed process: inventory every call, check hardware with shown math, probe the server, gate on evals, and compute cost only from your numbers. It will not mark a call ROUTE LOCAL without eval results or send sensitive data to the cloud.

    New in LLM Ops