- Home
- Skills
- Data & Databases
- data extraction cleaning structuring pipeline
Works with the AI Agents you already use
data extraction cleaning structuring pipeline
Automate the extraction, cleaning, and structuring of messy datasets and text into database-ready tables.
$19
data extraction cleaning structuring pipeline
Example session with this skill installed
I have a messy CSV named 'leads_raw.csv'. I need to clean it, remove duplicates, and structure it for a Salesforce import. The final file should be 'leads_clean.csv'.
- Read your context and instructions
- Compiled the data extraction cleaning
- Generated the data export
I have processed 'leads_raw.csv'. I standardized the phone formats, handled 14 missing email entries by flagging them, and removed 5 duplicate records.
Data Processing Pipeline Result
| Field | Strategy Applied | Status |
|---|---|---|
| Duplicate Removal | Exact match on email and full_name | 5 rows removed |
| Formatting | Standardized phone to E.164 and zip_code to 5-digit | Applied |
| Missing Data | Flagged rows with missing email in validation_error column | 14 rows flagged |
| Structure | Mapped to Salesforce Lead Import Header format | Complete |
| Export | Saved to leads_clean.csv | Success |
Processing Summary
- Input Rows: 150
- Cleaned Rows: 131
- Flagged Rows: 14 (Validation: Missing Email)
- Deleted Rows: 5 (Duplicates)
import pandas as pd
import numpy as np
def clean_leads(input_path, output_path):
try:
# Phase 1: Extract
df = pd.read_csv(input_path)
# Phase 2: Clean
df.drop_duplicates(subset=['email', 'full_name'], keep='first', inplace=True)
df['phone'] = df['phone'].astype(str).str.replace(r'\D', '', regex=True)
# Phase 3: Structure & Validate
df['validation_error'] = np.where(df['email'].isna(), 'Missing Email', '')
# Phase 5: Export
df.to_csv(output_path, index=False)
return "Success"
except Exception as e:
return f"Error: {str(e)}"
clean_leads('leads_raw.csv', 'leads_clean.csv')
Next steps
- Review the
validation_errorcolumn inleads_clean.csvfor the 14 flagged rows. - Upload the
leads_clean.csvfile directly to the Salesforce Data Import Wizard. - Verify that the phone number formatting matches your specific region requirements.
leads_raw.csv
CSV · data export
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
Messy datasets and unstructured notes stall analysis and break downstream integrations. This skill automates the extraction, cleaning, and structuring of raw data into production-ready formats.
What it does
- Source extraction reads data from CSV, Excel, APIs, or raw text notes using pandas.
- Data sanitization removes duplicates, fixes inconsistent formatting, and standardizes field values.
- Missing value handling applies configurable fill, drop, or flag strategies based on your requirements.
- Field extraction parses unstructured text to identify specific entities like dates, names, and amounts.
- Validation logging appends a reason column to flag specific rows that fail data integrity checks.
How it works
- Define Source Provide the input file or text and specify the desired destination.
- Configure Pipeline Select the cleaning strategy for missing data and the list of fields to extract.
- Run Transformation The agent executes the extract-clean-structure sequence with row-level error handling.
- Review Summary Inspect the processing summary and the exported file ready for CRM or database import.
Frameworks & tools
The pipeline is built on Python and pandas for high-performance data manipulation, utilizing try-except blocks for fault tolerance and row-level logging for auditability.
Why this beats prompting it yourself
Generic prompts often fail at edge cases like inconsistent date formats or partial null values. This skill enforces a multi-phase engineering pipeline with explicit validation steps, ensuring your output is actually importable into tools like Airtable or Notion.
Use cases
- Converting disorganized meeting notes into a structured Airtable-ready table.
- Normalizing messy customer CSV exports for CRM migration.
- Building a recurring pipeline to clean weekly API data drops.
- Extracting financial metrics from raw text logs into structured fields.
Known limitations
Requires Python environment with pandas installed. Does not support real-time streaming data or complex multi-table relational joining.
Requires Python and pandas. Best suited for batch processing rather than real-time streams. No support for complex relational joins across multiple tables.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 6 days ago
Requires
Permissions
No special permissions declared or detected