- Home
- Skills
- APIs & Backend
- llm client hardening
llm client hardening
Audit and fix LLM API integrations to prevent retry storms, hung requests, budget overruns, and API key leaks.
$14
Works with the AI Agents you already use
llm client hardening
Example session with this skill installed
Audit my OpenAI integration in app/services/llm.py. The requests sometimes hang for minutes and I think we are burning too many credits on failed calls.
- Read your context and instructions
- Compiled the llm client hardening
I have audited app/services/llm.py and identified 4 issues that explain the request hangs and credit burn. Your current configuration allows for a worst-case latency of 1,440 seconds (24 minutes) for a single request due to overlapping retry loops.
LLM Hardening Audit
| Severity | Finding | File:Line | Recommended Fix |
|---|---|---|---|
| High | Retries multiply (SDK x App) | app/services/llm.py:42 | Set max_retries=0 in SDK; manage in one loop. |
| High | No per-call timeout | app/services/llm.py:45 | Add explicit timeout=30.0 to the completion call. |
| Medium | Truncated output treated as OK | app/services/llm.py:58 | Raise error if finish_reason == "length". |
| Medium | No max_tokens cap | app/services/llm.py:45 | Set max_tokens to prevent 402/over-billing. |
Worst-case Latency: 4 app attempts x 3 SDK tries x 120s timeout = 1,440s
# Minimal diff for app/services/llm.py
client = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"),
max_retries=0 # Fix: One retry owner
)
def safe_generate(prompt):
try:
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
timeout=30.0, # Fix: Prevent hung workers
max_tokens=1000 # Fix: Budget cap
)
if response.choices[0].finish_reason == "length":
raise NonRetryableLLMError("Incomplete response") # Fix: Truncation guard
return response.choices[0].message.content
except Exception as e:
handle_llm_error(e)
Next steps
- Apply the retry consolidation and timeout fixes to
app/services/llm.py. - Deploy the
tests/test_llm_resilience.pysuite to verify backoff behavior. - Review the logs for
NonRetryableLLMErrorto identify prompts triggering the length limit.
Connects securely to your tools. The creator never sees your data.
About this skill
LLM API integrations often fail in ways standard HTTP calls do not. A single hung request can tie up a worker thread forever, and misconfigured retry logic can lead to "retry storms" that burn through your budget in minutes. This skill audits your backend code to ensure your LLM calls are production-ready.
What it does
- Leak detection identifies cross-user API key leaks and SSRF vulnerabilities in dynamic base URL configurations.
- Retry optimization calculates worst-case latency and consolidates conflicting retry loops between app code and SDKs.
- Truncation guarding prevents cut-off JSON or incomplete model responses from being treated as successful completions.
- Budget protection enforces
max_tokenscaps and prevents retrying errors that will never succeed, such as invalid keys or 400s. - Failure handling implements exponential backoff, jitter, and strict
Retry-Afterheader compliance for rate limits.
How it works
- Discovery scans the repository for call sites across OpenAI, Anthropic, Gemini, LangChain, and other providers.
- Audit runs a 12-point checklist to identify specific failures like missing timeouts or global SDK state risks.
- Reporting generates a severity-ranked findings table and calculates the theoretical worst-case latency of a single request.
- Remediation applies minimal, targeted diffs to fix approved issues while maintaining existing code style.
- Verification generates and runs a test suite to ensure fixes for transient errors, SSRF, and truncation logic persist.
Frameworks & tools
Supports Python and JavaScript/TypeScript SDKs including openai, anthropic, google-generativeai, litellm, and langchain. Handles FastAPI, Node.js, and server-side backend services.
Why this beats prompting it yourself
Generic prompts often miss the specific interplay between SDK default retries and application loops, leading to 20-minute request hangs. This skill uses a codified checklist and mathematical latency formulas that standard chat models ignore.
Use cases
- Auditing a multi-tenant AI SaaS where users bring their own API keys.
- Fixing a FastAPI backend where LLM calls are causing worker timeouts.
- Hardening an agentic workflow against silent JSON parsing errors.
- Securing a tool that allows custom API endpoints against SSRF.
Known limitations
Does not perform full penetration testing or solve for DNS rebinding gaps. Hardening is focused on client-side implementation rather than provider-side model behavior.
Does not solve for DNS rebinding. Focuses on client-side logic, not provider-side model behavior. Not a substitute for a full security penetration test.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 15 days ago
Permissions
Allowed Hosts