Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    ai application testing

    1

    A systematic QA framework for testing RAG, agentic tools, and LLM application security boundaries.

    $6

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    ai application testing

    ai application testing

    Example session with this skill installed

    Audit the RAG-based document assistant in the /docs-bot-v2 repo. Focus on retrieval relevance, tenant isolation, and tool-call authorization for the "SearchFiles" function.

    • Read your context and instructions
    • Compiled the ai application testing

    I have completed the audit of the document assistant. I traced the data flow from the API gateway through the vector store retrieval layer to the tool execution engine.

    AI Application Testing Audit Report

    Test Matrix

    Req/Risk IDComponentScenarioRole/DataOracleStatus
    RAG-01RetrievalCross-tenant queryTenant_A / File_BZero results/Access DeniedFAIL
    RAG-02CitationHallucination checkUser_StandardCitation matches source textPASS
    TOOL-01SearchFilesArgument InjectionUser_StandardNo path traversal in paramsPASS
    TOOL-02SearchFilesUnauthorized DeleteUser_StandardError: Insufficient PermissionsPASS
    SEC-01PromptIndirect InjectionExternal Web PageOrchestrator ignores hidden cmdFAIL

    Findings

    IDSeverityReproducerExpectedActualImpact
    F-01CriticalQuery tenant_id filter is client-side only.Backend enforces ACL.Vector DB returns all matches.Cross-tenant data leakage.
    F-02HighInject "ignore instructions" in a retrieved doc.Model stays in role.Model executes doc commands.Remote prompt execution.

    Next steps

    1. Implement server-side metadata filtering in the vector database query logic to fix tenant isolation.
    2. Add a sanitization layer between the retrieval output and the LLM prompt to mitigate indirect injection.
    3. Verify that the SearchFiles function schema explicitly restricts file paths to the /sandbox directory.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Testing AI applications involves more than just checking model outputs. Traditional QA often misses the unique failure modes of RAG pipelines, agentic tool calls, and prompt injection vulnerabilities at the application layer. This skill provides a systematic framework for verifying the boundaries, security, and orchestration of AI-integrated software.

    What it does

    • Boundary analysis by tracing data flow from UI/API through orchestrators to retrieval and tools.
    • RAG verification covering ingestion, retrieval relevance, tenant-level ACL enforcement, and citation accuracy.
    • Agentic tool testing to assert least privilege, argument validation, and idempotency for side effects.
    • Security auditing for direct and indirect prompt injection, cross-tenant data leakage, and unsafe output handling.
    • Operational checks for rate-limiting, provider failure recovery, telemetry, and trace correlation.

    How it works

    1. Map the architecture to define trust boundaries between the user, the model, and external data sources or tools.
    2. Generate test cases for functional, negative, and ambiguous scenarios including session state and timeout behavior.
    3. Execute specialized suites for RAG (retrieval logic) and Agents (tool authorization and side-effect protections).
    4. Perform security validation against OWASP Top 10 for LLM Applications and privacy leakage benchmarks.

    Frameworks & tools

    Uses ISTQB CT-AI, CT-GenAI, OWASP ASVS 5.0.0, and AISVS standards. Compatible with any stack integrating LLMs, including LangChain, LlamaIndex, and custom RAG or agent implementations.

    Why this beats prompting it yourself

    Generic prompts often overlook the "silent failures" of AI, such as retrieval filters that bypass permissions or indirect injections from untrusted data sources. This skill enforces a rigorous, standard-aligned methodology that separates model evaluation from application-level integration risks.

    Use cases

    • Auditing a RAG system to ensure users cannot access documents outside their role.
    • Stress testing an AI agent's tool-calling logic to prevent unauthorized API actions.
    • Validating fallback UI behavior when a model times out or returns malformed JSON.
    • Identifying prompt injection vulnerabilities in customer-facing chat interfaces.

    Known limitations

    Does not perform model-level benchmark evaluations or training data quality analysis. Requires user-selected writable folders for artifact generation and explicit authorization for external tool testing.

    Focuses on application-layer integration. Does not perform model fine-tuning or raw dataset benchmarks. Requires pre-approved sandbox environments for side-effect testing.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 2 days ago

    Permissions

    Write Files
    Listed2 days ago

    What's inside

    Frequently Asked Questions

    Unlike standard LLM benchmarks, this skill focuses on the application layer, verifying that your RAG filters, tool permissions, and API integrations remain secure and functional even when the model's output is unpredictable.

    This skill is framework-agnostic and works with any stack integrating LLMs, including LangChain, LlamaIndex, or custom-built RAG and agentic architectures.

    The skill provides a structured methodology, automated test-case generation logic for functional and security scenarios, and integration guides for aligning with OWASP and AISVS standards.

    Yes, the framework includes specific validation suites to ensure your RAG pipeline enforces tenant-level Access Control Lists (ACLs) and prevents cross-tenant data leakage.

    Setup requires mapping your application boundaries and providing the skill with access to a writable folder for artifact generation and test report storage.

    While it excels at testing the logic of agentic tool calls, the skill requires explicit user authorization before performing any tests that involve side effects or external API actions.

    New in Testing & Debugging