Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    rawfy

    1

    Convert any URL, PDF, or JavaScript-heavy site into clean, structured Markdown and JSON for AI agents.

    Free

    1 useSecurity scanned

    rawfy

    Example session with this skill installed

    Use the rawfy CLI to extract the main content from https://example.com/blog-post in markdown format and show me the link map.

    • Read your context and instructions
    • Compiled the rawfy

    I have extracted the content from the specified URL using the Rawfy CLI. The output has been cleaned and formatted as Markdown, including the metadata and link mapping for downstream processing.

    Extracted Content: https://example.com/blog-post

    FieldValue
    TitleThe Future of Agentic Workflows
    Formatmarkdown
    Word Count842 words
    Status200 OK

    Markdown Body

    
    # The Future of Agentic Workflows
    Published on October 24, 2023
    
    As AI agents move from simple chat interfaces to autonomous systems, the need for clean data extraction becomes critical. Most websites are designed for humans, not machines. 
    
    ## The Challenge of Noise
    Navigation bars, advertisements, and footer links create noise that confuses LLMs. By stripping these elements, we reduce token consumption by up to 60%.
    
    ## Structural Integrity
    Maintaining headers and lists in Markdown allows the agent to understand the document hierarchy without needing to parse complex CSS or nested divs.
    

    Extracted Links

    Next steps

    • Pass the Markdown Body to a summarization agent to generate a TL;DR.
    • Use the Extracted Links list to initiate a recursive crawl of the documentation section.
    • Save the output to a .md file for your local RAG knowledge base.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Web scraping for LLMs often results in noisy HTML, broken layouts, or empty responses from JavaScript-heavy sites. This skill provides a clean extraction layer that converts any URL into structured, machine-readable Markdown or JSON.

    What it does

    • Content extraction converts messy web pages into clean Markdown, JSON, or plain text.
    • Multi-format support handles standard HTML, JavaScript-rendered SPAs, and PDF files natively.
    • Structured output returns a consistent object containing the page title, body content, metadata, and a link map.
    • Interface flexibility provides access via CLI, MCP server, REST API, Node.js SDK, or Python wrapper.
    • Agentic pipeline integration enables research loops by fetching content and passing structured data to downstream summarizers or analyzers.

    How it works

    1. Selection Choose the interface that matches your environment, such as the CLI for terminal work or MCP for agentic tool calls.
    2. Extraction Run the fetch command with the target URL and optional flags for format or JavaScript waiting.
    3. Processing The tool parses the DOM, extracts core content, and strips out noise like ads or navigation.
    4. Integration Use the resulting structured JSON or Markdown directly in your LLM context or database.

    Frameworks & tools

    Built for Node.js environments and Python wrappers. It integrates with MCP-compatible agents (like Claude Desktop) and provides a REST API for microservice architectures.

    Why this beats prompting it yourself

    Writing custom scrapers for every site is brittle and time-consuming. This skill handles the edge cases of modern web tech, like SPAs and PDFs, delivering pre-cleaned data that saves token costs and prevents model hallucinations caused by HTML noise.

    Use cases

    • Research agents fetching web documentation to generate technical summaries.
    • Link crawlers extracting link arrays to map out website architectures.
    • Data extraction turning product pages into structured JSON for price tracking.
    • Knowledge retrieval converting PDFs and articles into Markdown for RAG systems.

    Known limitations

    Does not support local file paths or direct API endpoints that already return JSON. JS-heavy sites may require specific wait flags to ensure full content rendering.

    Does not support local file paths. Direct JSON APIs should be called natively. JS-heavy SPAs may require --js or --wait flags.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    1 use

    Times used to date

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 1 month ago

    • Free with an Agensi account

    Requires

    GitHub
    Shields
    Nodejs
    Wikipedia
    Iana
    Spa Example
    Url1
    Url2
    rawfy
    Python

    Permissions

    Terminal / Shell
    Browser
    Network Access

    Allowed Hosts

    example.com
    ...
    img.shields.io
    nodejs.org
    en.wikipedia.org
    www.iana.org
    github.com
    spa-example.com
    url1.com
    url2.com
    bad-url

    File Scopes

    references/**
    Listed1 month ago

    What's inside

    Frequently Asked Questions

    rawfy converts complex URLs, JavaScript-heavy SPAs, and PDFs into clean Markdown or JSON, removing ads and navigation noise to reduce LLM token costs and prevent hallucinations.

    It is fully compatible with MCP-enabled agents like Claude Desktop, and can be integrated into any environment via its REST API, Node.js SDK, or Python wrapper.

    Your purchase includes the core extraction engine, the CLI tool, the MCP server for agentic tool calling, and full access to the Node.js and Python integration libraries.

    Yes, for sites that load content dynamically, you can use specific 'wait' flags to ensure the tool captures the fully rendered page before processing the content.

    While rawfy excels at public web pages and PDFs, it does not currently support local file path ingestion or sites that already serve raw JSON data.

    New in APIs & Backend