llm client hardening

    1

    Audit and fix LLM API integrations to prevent retry storms, hung requests, budget overruns, and API key leaks.

    $14

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    llm client hardening

    Example session with this skill installed

    Audit my OpenAI integration in app/services/llm.py. The requests sometimes hang for minutes and I think we are burning too many credits on failed calls.

    • Read your context and instructions
    • Compiled the llm client hardening

    I have audited app/services/llm.py and identified 4 issues that explain the request hangs and credit burn. Your current configuration allows for a worst-case latency of 1,440 seconds (24 minutes) for a single request due to overlapping retry loops.

    LLM Hardening Audit

    SeverityFindingFile:LineRecommended Fix
    HighRetries multiply (SDK x App)app/services/llm.py:42Set max_retries=0 in SDK; manage in one loop.
    HighNo per-call timeoutapp/services/llm.py:45Add explicit timeout=30.0 to the completion call.
    MediumTruncated output treated as OKapp/services/llm.py:58Raise error if finish_reason == "length".
    MediumNo max_tokens capapp/services/llm.py:45Set max_tokens to prevent 402/over-billing.

    Worst-case Latency: 4 app attempts x 3 SDK tries x 120s timeout = 1,440s

    
    # Minimal diff for app/services/llm.py
    client = OpenAI(
        api_key=os.environ.get("OPENAI_API_KEY"),
        max_retries=0  # Fix: One retry owner
    )
    
    def safe_generate(prompt):
        try:
            response = client.chat.completions.create(
                model="gpt-4o",
                messages=[{"role": "user", "content": prompt}],
                timeout=30.0, # Fix: Prevent hung workers
                max_tokens=1000 # Fix: Budget cap
            )
            if response.choices[0].finish_reason == "length":
                raise NonRetryableLLMError("Incomplete response") # Fix: Truncation guard
            return response.choices[0].message.content
        except Exception as e:
            handle_llm_error(e)
    

    Next steps

    1. Apply the retry consolidation and timeout fixes to app/services/llm.py.
    2. Deploy the tests/test_llm_resilience.py suite to verify backoff behavior.
    3. Review the logs for NonRetryableLLMError to identify prompts triggering the length limit.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    LLM API integrations often fail in ways standard HTTP calls do not. A single hung request can tie up a worker thread forever, and misconfigured retry logic can lead to "retry storms" that burn through your budget in minutes. This skill audits your backend code to ensure your LLM calls are production-ready.

    What it does

    • Leak detection identifies cross-user API key leaks and SSRF vulnerabilities in dynamic base URL configurations.
    • Retry optimization calculates worst-case latency and consolidates conflicting retry loops between app code and SDKs.
    • Truncation guarding prevents cut-off JSON or incomplete model responses from being treated as successful completions.
    • Budget protection enforces max_tokens caps and prevents retrying errors that will never succeed, such as invalid keys or 400s.
    • Failure handling implements exponential backoff, jitter, and strict Retry-After header compliance for rate limits.

    How it works

    1. Discovery scans the repository for call sites across OpenAI, Anthropic, Gemini, LangChain, and other providers.
    2. Audit runs a 12-point checklist to identify specific failures like missing timeouts or global SDK state risks.
    3. Reporting generates a severity-ranked findings table and calculates the theoretical worst-case latency of a single request.
    4. Remediation applies minimal, targeted diffs to fix approved issues while maintaining existing code style.
    5. Verification generates and runs a test suite to ensure fixes for transient errors, SSRF, and truncation logic persist.

    Frameworks & tools

    Supports Python and JavaScript/TypeScript SDKs including openai, anthropic, google-generativeai, litellm, and langchain. Handles FastAPI, Node.js, and server-side backend services.

    Why this beats prompting it yourself

    Generic prompts often miss the specific interplay between SDK default retries and application loops, leading to 20-minute request hangs. This skill uses a codified checklist and mathematical latency formulas that standard chat models ignore.

    Use cases

    • Auditing a multi-tenant AI SaaS where users bring their own API keys.
    • Fixing a FastAPI backend where LLM calls are causing worker timeouts.
    • Hardening an agentic workflow against silent JSON parsing errors.
    • Securing a tool that allows custom API endpoints against SSRF.

    Known limitations

    Does not perform full penetration testing or solve for DNS rebinding gaps. Hardening is focused on client-side implementation rather than provider-side model behavior.

    Does not solve for DNS rebinding. Focuses on client-side logic, not provider-side model behavior. Not a substitute for a full security penetration test.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 15 days ago

    Permissions

    Network Access
    Write Files

    Allowed Hosts

    api.
    Listed15 days ago

    What's inside

    Frequently Asked Questions

    This skill identifies and resolves 'retry storms'—where conflicting retry loops between your app and an SDK cause runaway costs—and 'worker hangs' caused by missing timeouts in long-running LLM streams.

    The skill supports Python and JavaScript/TypeScript environments, covering popular SDKs like OpenAI, Anthropic, Gemini, LiteLLM, and frameworks like LangChain and FastAPI.

    You receive a detailed 12-point audit report, a severity-ranked findings table, a suite of targeted code diffs for remediation, and a custom test suite to prevent regressions.

    Yes, it includes specialized leak detection for multi-tenant applications to ensure user-provided API keys are isolated and that dynamic base URLs are protected against SSRF vulnerabilities.

    Unlike generic prompts, this skill uses mathematical latency formulas and a codified checklist to address the specific, often hidden interplay between SDK defaults and application-level logic.

    The skill focuses on hardening the client-side integration and SDK implementation; it does not perform full-scale penetration testing or address provider-side model hallucinations.

    New in APIs & Backend