Works with the AI tools you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIGitHub CopilotGitHub CopilotGemini CLIGemini CLI+20 more

    RAG Knowledge Base Design and QA Suite

    1

    Design optimized chunking strategies, metadata schemas, and hallucination-proof QA suites for production RAG pipelines.

    $5

    Secure checkout via Stripe

    30-day refund guarantee

    Converts to your local currency at checkout

    0 installsSecurity scanned
    RAG Knowledge Base Design and QA Suite

    RAG Knowledge Base Design and QA Suite

    Example session with this skill installed

    I'm building a RAG bot for our internal HR policies (PDFs). The retrieval is messy and it keeps making up vacation rules. Design a chunking strategy and a QA suite to fix this.

    • Read your context and instructions
    • Compiled the rag knowledge base
    • Generated the document

    I have designed a RAG architecture and evaluation suite tailored for HR policy documents to ensure retrieval precision and eliminate hallucinations.

    RAG Knowledge-Base QA Report

    🏗️ Chunking & Metadata Strategy

    • Chunk Size: 512 tokens (optimal for granular policy clauses).
    • Overlap: 15% (approx 75 tokens) to maintain context across paragraph breaks.

    Metadata Schema: {"policy_type": "string", "effective_date": "date", "department": "string", "source_file": "string", "page_number": "int"}. This allows filtering by department to prevent cross-office policy confusion.

    🔍 Retrieval Test Cases (Golden Queries)

    Query TypeQuery ExampleExpected Top Chunk
    Simple Factual"What is the annual leave entitlement for senior managers?"Chunk from vacation_policy.pdf containing the grade-based entitlement table.
    Complex Multi-hop"Can I carry over unused leave if I joined after July?"Intersection of accrual_rules.pdf and prorated_benefits_clause.pdf.
    Negative/Out-of-Scope"What is the company's stock price today?"Zero relevant chunks / No HR policy matches.

    🛡️ Hallucination & Citation System Prompt

    CRITICAL: You must answer ONLY using the provided context. If the answer is not explicitly in the context, reply exactly with: 'I cannot find this information in the provided HR documents.' You MUST include inline citations for every claim in the format [Source: source_file, Page: page_number].

    ⚡ Optimization Notes

    • Metadata Filtering: Always filter by department before similarity search to narrow the vector space.

    Reranking: Implement a Cross-Encoder reranker (like Cohere or BGE-Reranker) to evaluate the top 10 retrieved chunks before passing the top 3 to the LLM.

    Next steps

    1. Update your ingestion script with the 512/15% chunking parameters.
    2. Implement the metadata schema in your vector database index.
    3. Run the Golden Queries through your retrieval pipeline to calculate Hit Rate and MRR.

    rag-knowledge-base-design-and-qa-suite.pdf

    PDF · document

    Generated

    Example file from a real run - the skill writes it into your workspace.

    Connects securely to your tools. The creator never sees your data.

    What you get

    Optimize chunking and overlap for specific document types and structures.Eliminate hallucinations with strict context-only system prompt snippets.Benchmarking retrieval accuracy using multi-hop and negative golden queries.Design metadata schemas for advanced hybrid search and filtering.

    About this skill

    The problem

    RAG pipelines often fail due to poor retrieval, lack of citations, and unpredictable hallucinations. Standard "just feed the context" prompts frequently lead to the AI making up information when the database returns irrelevant chunks.

    What it does

    • Defines precise chunk sizes and overlap percentages based on specific document types.
    • Structures metadata schemas to enable efficient hybrid search and filtering.
    • Generates "Golden Query" test cases including multi-hop and negative queries to audit vector DB performance.
    • Drafts strict system prompt snippets for context-only answering and mandatory inline citations.
    • Recommends reranking and filtering strategies to reduce token waste and latency.

    Frameworks & tools

    Works with vector databases like Pinecone, Weaviate, Milvus, and ChromaDB. Applicable to RAG frameworks including LangChain, LlamaIndex, and Haystack.

    Why this beats prompting it yourself

    Generic prompts don't solve underlying data architecture issues. This skill provides the technical specifications for the data layer and the evaluation framework needed to prove your RAG pipeline actually works before you hit production.

    Use cases

    • Designing the chunking strategy for a massive corpus of technical PDF manuals.
    • Reducing hallucination rates in a customer support bot by enforcing context-only rules.
    • Benchmarking retrieval accuracy for a new vector database implementation.
    • Optimizing token costs by refining metadata filters and context window usage.

    How to install

    Works the same in every agent - Claude, Cursor, Codex, Copilot and 20+ more.

    ~30 seconds
    1. 1

      Download the ZIP

      Free skills download straight away. Paid skills unlock right after purchase.

    2. 2

      Unzip into your skills folder

      Every agent reads skills from one folder on your machine. Drop the unzipped folder in there.

    3. 3

      Ask your agent to use it

      Restart the agent if it was already running. It picks the skill up automatically - no config needed.

    Skills folder by agent

    Click the path to copy it. Create the folder if it does not exist yet.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill passes our trust checks below.

    Security scanned

    Passed our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    30-day refund

    Not a fit? Get your money back

    Trust & safety

    Security scanned

    Verified clean 14 days ago

    • Passed all security checks, Safe to install

    Listed14 days ago

    What's inside

    Frequently Asked Questions