- Home
- Skills
- APIs & Backend
- site email finder
Works with the AI Agents you already use
site email finder
Maintain and extend a modular Python web crawler for CSV-based email extraction and contact discovery.
$1.50
site email finder
Example session with this skill installed
I need to add a new CLI flag called --timeout to control the HTTP request duration in extractor.py and ensure it is configurable via the CLI.
- Read your context and instructions
- Compiled the site email finder
I have implemented the --timeout flag across the project layers. This involved updating config.py for defaults, extractor.py to respect the timeout during fetching, and cli.py to expose the argument to the user.
Implementation Details
| Module | Change Description |
|---|---|
src/site_email_finder/config.py | Added DEFAULT_TIMEOUT = 30 to the configuration constants. |
src/site_email_finder/extractor.py | Updated fetch_page and EmailExtractor to accept a timeout parameter in the requests call. |
src/site_email_finder/cli.py | Added --timeout argument to the parser and passed the value through to the extractor instance. |
tests/test_extractor.py | Added a test case to verify that the timeout parameter is correctly passed to the underlying HTTP client. |
# src/site_email_finder/extractor.py (partial)
def fetch_page(url, timeout=DEFAULT_TIMEOUT):
try:
response = requests.get(url, timeout=timeout, headers=USER_AGENT_HEADERS)
response.raise_for_status()
return response.text
except requests.exceptions.Timeout:
# Differentiate timeout from other connection errors
raise ConnectionError(f"Request timed out after {timeout} seconds")
Next steps
- Run
pytest tests/test_extractor.pyto confirm the new timeout logic passes. - Verify the CLI help text by running
python -m site_email_finder.cli --help. - Update
README.mdto document the new--timeoutflag for users.
Connects securely to your tools. The creator never sees your data.
About this skill
Managing a custom web crawler for contact discovery often leads to spaghetti code where I/O, networking, and CLI logic become inseparable. This skill provides a structured environment for maintaining and extending a Python-based website crawler that extracts business emails from CSV sources. It enforces a strict separation between HTTP fetching, CSV field mapping, and command-line orchestration.
What it does
- Extract Emails through URL normalization, contact-page discovery, and junk-filtering logic.
- Map CSV Fields to handle arbitrary input schemas via CLI overrides or YAML configurations.
- Manage Crawl State by preserving resumability and distinguishing between missing emails and connection errors.
- Maintain Modular Code by keeping extraction logic independent of I/O and CLI concerns.
- Validate Changes using structured testing patterns with mocked HTTP responses.
How it works
- Analyze Project Layers by scanning the specialized modules for extraction, I/O, or CLI logic.
- Configure Mappings via CLI flags or YAML to align input CSV columns with crawler requirements.
- Execute Crawls with built-in retry handling and progress logging.
- Run Tests using
pytestandresponsesto verify behavior without making real network calls.
Frameworks & tools
This skill is built for Python 3.9+ environments utilizing pytest for testing, responses for HTTP mocking, and ruff for linting and formatting.
Why this beats prompting it yourself
Generic coding assistants often mix side effects with pure logic, breaking the crawler's resumability or cluttering the extractor with CLI prints. This skill understands the specific architectural boundaries required to keep the crawler scalable and testable.
Use cases
- Add a new contact-page discovery strategy without breaking the core extractor.
- Implement custom CSV field mappings for a specific lead list schema.
- Debug connection retry logic and error classification for large-scale crawls.
- Refactor the I/O layer to support new output formats while maintaining resume support.
Known limitations
Requires Python 3.9 or newer. Testing suite requires the responses library and explicitly avoids real network access.
Requires Python 3.9+. Mocked HTTP testing only. Does not support real network access during automated test runs.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 1 day ago
Requires