Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Near-Duplicate Text Detector

    1

    This skill measures overlap the same way every time and gives a clear verdict with a receipt.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Near-Duplicate Text Detector

    Near-Duplicate Text Detector

    Example session with this skill installed

    I asked you to write a fresh help-centre intro, but draft.txt reads a lot like our old one (orig.txt). How close is it? Also compare orig.txt with other.txt, which we wrote from scratch.

    • Read your context and instructions
    • Compiled the near-duplicate text detector

    I ran near-duplicate-check with --k 3 (better for short paragraphs) on both pairs.

    orig.txt vs draft.txt

    {
      "verdict": "NEAR-DUPLICATE",
      "similarity": 0.796875,
      "receipt": {
    
    "algorithm": "kar-shingle-minhash-v1", "k": 3, "n": 64,
    
        "docA": { "shingleCount": 54,
          "normalizedSha256": "250815fb87514efaa1fbc5b61be39a41658d7bf5fe4511e011d4927056214ebe" },
        "docB": { "shingleCount": 54,
          "normalizedSha256": "b1fd5de07d15b128cfb781de7cb491e5af16a84ba317df2deed3a7350e81937a" },
    
    "similarityMatches": 51, "similarityTotal": 64, "verdict": "NEAR-DUPLICATE"
    
      },
      "receiptSha256": "83a5ecdadbb23e1667df8444b5185f5f219974f78f289393c4d361aaa1889b46"
    }
    

    orig.txt vs other.txt (default k=5)

    {
      "verdict": "DISTINCT",
      "similarity": 0,
      "receipt": { "algorithm": "kar-shingle-minhash-v1", "k": 5, "n": 64,
        "similarityMatches": 0, "similarityTotal": 64, "verdict": "DISTINCT" },
      "receiptSha256": "56ef6508ddf6e564bf40a437b03fa02b7844117df8dc174a0a93ea0d9657fe92"
    }
    

    Summary: draft.txt is a near-duplicate of the old intro (about 80% estimated similarity). Only two words were swapped ("Start" became "Begin", "turn on" became "switch on"). I'll rewrite it properly. other.txt shares no 5-word runs with the original, so it is genuinely new.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Agent Trust: check how close a draft is to its source

    Agents that summarise or rewrite can stay too close to the original, and content teams recycle the same paragraphs. This skill measures overlap the same way every time and gives a clear verdict with a receipt.

    What it does

    • Normalises both texts and breaks them into overlapping word shingles (5 words by default; adjustable with --k)
    • Builds a deterministic MinHash signature (64 hash functions by default; --n)
    • Returns a similarity ratio and a verdict:
      • IDENTICAL: the normalised texts are the same
      • NEAR-DUPLICATE: similarity 0.75 or higher
      • OVERLAPPING: 0.35 to 0.75
      • DISTINCT: below 0.35

    When an agent should use it

    • Checking whether a draft or AI output stays too close to a source
    • De-duplicating tickets, docs, FAQs or product descriptions
    • Comparing two versions of a text to see how much really changed

    What it outputs

    JSON: verdict, similarity ratio, and a receipt with shingle counts, hashes of each normalised text and signature, match counts, and a receipt SHA-256.

    Limitations

    • It measures word overlap, not meaning. A full rewrite with the same ideas will score as DISTINCT.
    • Shingle size matters. With the default of 5 words, a few changed words in a short paragraph can drop a near-copy to OVERLAPPING; use --k 3 for short texts.
    • MinHash is an estimate; with 64 hashes the ratio moves in steps of 1/64.
    • Not a plagiarism database: it only compares the two texts you give it.
    • The exit code is 0 for every verdict, so agents and scripts should read verdict rather than the exit code.

    Open source, packaged skill

    Original work by Simon Gant. The core logic is public under the MIT licence (kar-shingle). This listing is the packaged, tested agent skill: SKILL.md workflow, CLI, receipt output and docs. Requires Node.js 18+, zero dependencies, no network, no API keys.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed today

    Requires

    Node.js 18+
    Node.js

    Permissions

    Runs one local Node.js script (`node scripts/near-duplicate-check.js`) through the agent's shell tool. Reads only the two text files you pass it. No network access, no file writes, no environment variables read. Pure compute.

    Listedtoday

    What's inside

    Frequently Asked Questions

    Not against the web. It compares the two texts you give it, offline.

    With 5-word shingles, a couple of changed words in a short text break many shingles. Use `--k 3` for short paragraphs.

    No. It measures word overlap, which is what makes it repeatable. A rewrite with the same ideas in new words will score low.

    Hashes of each normalised text and MinHash signature, plus the settings, so anyone can re-run it and get the same numbers.

    The core logic is MIT open source. You are paying for the packaged, tested agent skill and workflow.