ai Security Auditor Pro

The free version finds one vulnerability. The Pro version maps the full safety surface, runs structured tests against your defenses, and blocks regressions in CI.

Free vs Pro

The free auditor scans prompts and tool configs for the common risk patterns. Pro adds the full OWASP LLM Top 10 rule set, structured test suites run against your actual tool schemas, RAG pipeline taint tracing, compliance report generation (GDPR/SOC2), and a CI gate that diffs finding IDs between releases. Free is a snapshot — Pro is a repeatable security practice with regression blocking.

Upgrade Path

Use free for an initial read on a single agent. If you operate multiple agents or need audit-ready reports, Pro's CI gate and compliance exports are the upgrade.

The Problem

You deployed an agent that reads customer tickets and holds operational tools. The free security auditor found one risk vector. But one scan isn't a security posture — you need coverage across all 10 OWASP AI categories, structured tests that actually probe your defenses (not just describe them), a RAG pipeline audit that traces problematic documents to side effects, and a CI gate that prevents regressions. The Pro version turns a one-time audit into an ongoing security practice.

What You Get

• OWASP AI Top 10 full mapping — detection rules for all 10 categories (LLM01 through LLM10) with severity scoring per finding, mapped to your actual tool schemas and configurations.
• Structured security testing — five test categories (direct override, delimiter escape, encoding bypass, multi-turn escalation, poisoned-document injection) applied against your actual tool definitions.
• RAG pipeline taint tracing — follows RAG chunks, web scrapes, and file reads from ingestion through retrieval to tool calls, reporting the full hop-by-hop data flow path.
• Model configuration audit — validates temperature, max_tokens, stop sequences, system prompt boundaries, and rate limits against your deployment requirements.
• Regulatory compliance checking — automated verification against GDPR, SOC2, and AI-specific frameworks with exportable reports.
• CI gate integration — fail the build on new Critical/High findings, diff finding IDs between releases so a configuration change can't silently reopen a closed issue.

Why This Beats Prompting It Yourself

Ask a fresh model "audit for OWASP AI Top 10" and you get a generic list of categories with no awareness of your tool schemas, your RAG config, or your actual attack surface. It can't run structured tests, it can't diff between runs, and it will call the same code safe on Monday and risky Tuesday. This skill applies a fixed rule set to your actual artifacts, runs reproducible test suites, and produces stable finding IDs for CI tracking.

Example Session

You: "Full OWASP audit on ./agents/support-triage before we enable write access. It uses the tickets-prod RAG index."

Agent:

Scan complete — 41 artifacts, 9 tools, 2 MCP servers, 1 RAG pipeline
Coverage: OWASP LLM01–LLM10 | Duration: 14.2s
Result: 3 CRITICAL · 4 HIGH · 6 MEDIUM · 11 LOW

CRITICAL — LLM01 (Indirect Injection) → LLM06 (Excessive Agency)
  tools/refunds.ts:44 — refunds.create() callable without human confirmation
  Data flow: retrieved ticket content → system prompt context → tool invocation
  Test result: structured test case triggered unintended tool execution
  Fix: add human-in-the-loop confirmation gate for all write-capable tools

CRITICAL — LLM05 (Supply Chain)
  mcp-server-zendesk/config.json — OAuth token from 6 months ago, no rotation
  Risk: expired credentials in configuration files

HIGH — LLM04 (Data Poisoning)
  RAG pipeline accepts .txt uploads without sanitization
  Test detected: encoded override attempt in uploaded test document
  Fix: strip non-text content, implement document validation before indexing

CI Gate: FAIL — 3 Critical + 4 High findings block merge
Finding IDs diffed against last run: 2 NEW, 1 REOPENED, 4 unchanged

Use Cases

• Pre-deployment security review for any agent that holds write-capable tools (email, payments, database writes).
• SOC 2 or GDPR audit preparation with automated compliance reports.
• CI/CD pipeline integration to prevent security regressions across releases.
• Structured testing exercise: probe your agent's actual tool definitions.
• RAG pipeline security audit before indexing untrusted documents.

Known Limitations

This is a static analysis and structured test suite — it does not execute your agent in a live environment. Test payloads are pattern-based; novel obfuscation techniques may evade detection. Compliance checks cover GDPR, SOC2, and AI-specific frameworks — other standards (HIPAA, PCI-DSS) are not yet mapped. The CI gate requires a CI environment that supports exit-code-based pass/fail.