- Home
- Skills
- Data & Databases
- data-pipeline-agent
Works with the AI Agents you already use
Data Pipeline Agent
Turn any API or CSV into a ready-to-query SQLite database in minutes. Zero-config schema inference, type validation, and optional RAG vectors.
$12
data-pipeline-agent
Example session with this skill installed
I need to sync my product catalog from a CSV file at './imports/products.csv' into a SQLite db. I also need embeddings for the 'description' column using OpenAI.
- Read your context and instructions
- Compiled the data-pipeline-agent
- Generated the data export
I have configured the data pipeline for your product catalog. I created the necessary configuration file, inferred the schema from your CSV, and prepared the ingestion scripts. ## Data Pipeline Configuration ```yaml source: type: csv path: ./imports/products.csv destination: database: ./data/inventory.db table: products embeddings: enabled: true model: openai columns: [description] sync: mode: incremental key_column: sku updated_at_column: last_modified ``` I have also generated the `schema.json` by inspecting the CSV headers and data types. **Execution Commands:** 1. Run ingestion: `python scripts/ingest.py --config config.yaml --schema schema.json` 2. Generate embeddings: `python scripts/embed.py --config config.yaml --database ./data/inventory.db` **Next steps** - Add your `OPENAI_API_KEY` to the `.env` file. - Run the ingestion script to populate `inventory.db`. - Execute the embedding script to build the vector search table.
data-pipeline-agent.csv
CSV · data export
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Local data pipelines for RAG or analytics become brittle scripts that fail on schema changes. You waste time on type validation, sync, and embeddings boilerplate.
What it does
- Ingests APIs, CSVs, and Google Sheets into SQLite
- Auto-infers schema and validates types
- Incremental sync via primary keys and updated_at
- Optional embeddings for text columns (local or OpenAI)
- Retry with exponential backoff
Why it beats prompting yourself
Generic prompts miss edge cases: database locking, rate limiting, schema evolution. Ships dedicated scripts for inference, ingestion, validation.
Use cases
- Sync CRM or CMS into SQLite for RAG
- Self-updating analytics dashboard from CSV exports
- Local knowledge base for AI agents with auto-embeddings
- ETL prototype without heavyweight tools
Limitations
Not for real-time streaming, cloud warehouses, or binary data.
Requires Python 3.9+. Not designed for real-time streaming (Kafka/Flink) or massive data warehouses like Snowflake. Structured data only.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 2 months ago
Requires