- Home
- Skills
- Data & Databases
- job posting deduper
More screenshots
Works with the AI Agents you already use
job posting deduper
**The same SOC Analyst job showed up three times this week.
$8.29
About this skill
The same SOC Analyst job showed up three times this week. Apply once, on the right site.
Give it the CSVs you exported or copied from each board. It maps their different column names, cleans URLs (drops utm_*, refId, trackingId and similar parameters, and reads LinkedIn job IDs and Indeed jk keys), and normalizes companies (Inc/LLC), titles (Sr./Senior, I/1, Tier 1, '(Remote)') and locations (Austin, Texas, United States = Austin, TX). Jobs merge on the same URL, the same company + title + location, or near-identical descriptions from the same company. For each job it keeps the best apply link (the employer's careers or ATS page such as Workday, Greenhouse or Lever over aggregators), the salary from whichever copy had one, and the earliest posted date. It shows board overlap so you can see which boards add unique jobs, and marks or hides jobs already in your application tracker.
What it does
- Column aliases for common exports (Job Title, Employer, Link, Date Posted...)
- URL cleaning plus LinkedIn/Indeed job-ID matching
- Title, company and location normalization
- Union-find merging on URL, company+title+location, or description similarity
- Canonical link prefers the employer/ATS page; keeps salary and earliest date
- Board-overlap table and --applied / --hide-applied using your tracker CSV
Who it's for
Job seekers pulling from several boards and alerts who want one deduplicated list to work from, and anyone feeding a bulk-tailoring workflow.
Limits
Needs title and company in every file. Without descriptions, differently titled duplicates may not merge. Location normalization covers US formats best.
What's included
SKILL.md: workflow, commands, how to explain resultsscripts/: the tool (standard-library Python)references/: sourcesexamples/: sample inputs used in the demo (all made up)tests/: unit tests (Python 3.8 / 3.9 / 3.13)
Standard-library Python only. No API keys, no network calls, no installs.
- Needs title and company in every file. Without descriptions, differently titled duplicates may not merge.
- Location normalization covers US formats best.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed today
Permissions
File Scopes
Fully offline. No network calls, no credential access, and it never logs into, scrapes or submits anything on job boards.