More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    πŸ€– Robots Sitemap Validator

    2

    It automates the tedious process of auditing local sitemap and robots.txt files, ensuring your site's architecture is search-engine friendly.

    Free

    5 usesSecurity scanned
    πŸ€– Robots Sitemap Validator

    πŸ€– Robots Sitemap Validator

    Example session with this skill installed

    Audit the robots.txt and sitemap.xml in my public directory to make sure I'm not blocking search engines from the main site.

    • Read your context and instructions
    • Compiled the robots sitemap validator

    Scan results for /public

    • [CRITICAL] robots.txt L4: User-agent: * Disallow: / (Blocks all crawling)
    • [WARNING] robots.txt L12: Sitemap: /sitemap_index.xml (File not found in local directory)
    • [INFO] sitemap.xml: Valid XML structure found with 42 URLs.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Ensure Search Engines Can Crawl Your Code

    The Robots Sitemap Validator is a specialized tool for developers and SEO teams to catch critical crawling mistakes before they hit production. It automates the tedious process of auditing local sitemap and robots.txt files, ensuring your site's architecture is search-engine friendly.

    What it does

    • Identifies syntax errors in robots.txt and sitemap.xml files.
    • Detects "orphan" sitemaps where a robots.txt references a file that doesn't exist locally.
    • Flags high-severity blocking rules that might accidentally de-index important sections of your site.
    • Provides structured feedback including rule IDs, severity levels, and exact line numbers.

    Why use this skill?

    Manual SEO audits are prone to human error, and production-only crawlers find issues too late. This skill integrates into your local development or CI/CD workflow to detect issues while you're coding. It supports standard web protocols and works with any directory structure, providing a layer of safety that generic LLM prompting cannot match in terms of precision and file-system awareness.

    It validates the files you give it, so it cannot confirm that a referenced sitemap exists unless that file is provided alongside (it flags those for review). It does not fetch the live site, and the URL-count and size checks are guidance notes against the documented limits.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    5 uses

    Times used to date

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 5 months ago

    • Free with an Agensi account

    Permissions

    Terminal / Shell
    Read Files

    File Scopes

    **/robots.txt
    **/sitemap*.xml

    Notes: Read-only. Also accepts pasted text via --stdin. Prints findings with rule id, severity, file, and line. It does not crawl, render JavaScript, submit to search engines, or read environment variables.

    Listed5 months ago

    What's inside

    Frequently Asked Questions

    This skill identifies critical errors like site-wide 'Disallow' rules, malformed XML syntax, missing sitemap declarations, and size limit violations (exceeding 50,000 URLs or 50MB) before you deploy.

    No, the validator operates entirely by reading local files or pasted text; it never pings your live server, making it safe for auditing staging environments or pre-production code.

    The skill is compatible with any AI agent that has file-system or text-input capabilities, allowing it to be integrated into your local development environment or CI/CD pipelines.

    You receive the full skill logic along with documentation on how to trigger the validation via direct file upload or text input for instant structured feedback.

    Yes, the tool is designed to catch mixed HTTP/HTTPS schemes within your <loc> tags and ensures that all URLs include a mandatory transport protocol.

    New in Marketing & SEO