Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    ETL Pipeline Builder — Design Cleaner Data Transformation

    1

    Design explicit, testable ETL and ELT workflows with defined grain, safe reruns, data-quality checks, reconciliation and recovery behavior.

    $9.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    ETL Pipeline Builder — Design Cleaner Data Transformation

    ETL Pipeline Builder — Design Cleaner Data Transformation

    Example session with this skill installed

    Illustrative scenario: A daily CSV contains order_id and amount. order_id should be unique; amount must be a valid nonnegative decimal. No warehouse engine or incremental timestamp exists. Design a safe snapshot pipeline that quarantines invalid records and avoids silently choosing duplicate orders.

    • Read your context and instructions
    • Compiled the etl pipeline builder

    Contract: One row per order_id in each supplied daily snapshot. amount is a nonnegative decimal; allowed precision and currency must be confirmed. Snapshot completeness and deletion meaning remain unknown.

    1. Landing: Preserve the original authorized file with batch ID and checksum for diagnosis and replay. Reject unreadable files without publishing partial output.
    2. Staging: Parse both fields explicitly. Keep raw values and validation reasons. Treat missing IDs, invalid decimals and negative amounts as failures.
    3. Duplicate handling: Detect repeated order_id values. Quarantine all rows for the ambiguous key or stop the batch under the agreed policy; never select an arbitrary winner.
    4. Validation: Report landed, valid and quarantined counts. Reconcile landed = valid + quarantined under exclusive classification. Validate unique keys and nonnegative amounts in the accepted set. Compare totals only for comparable populations.
    5. Publication: Publish a versioned validated snapshot only after the agreed quality gate. Exact transactional replacement or pointer-swap behavior depends on the target engine. Retain the previous successful version for rollback.
    6. Reruns: Same file checksum must not create duplicate published snapshots. Use a controlled batch manifest and replay policy.
    7. Observability: Record source file, batch ID, counts, failures, status and target version. Alert on failures only through configured authorized tooling.

    Tests: Valid file, missing ID, duplicate ID, malformed/negative amount, empty file, failed publication and identical rerun. Verify source-to-target reconciliation and rollback on representative data.

    Unknowns: Currency/precision, snapshot completeness, deletion semantics, target transactions, storage and retention. No incremental cursor invented.

    Execution: Design only; no files loaded, tests run or deployment completed.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    ETL Pipeline Builder helps data and analytics engineers design or review source-to-target transformation workflows. It clarifies business meaning, grain, keys, schema contracts and consumer expectations before proposing extraction, staging, cleansing, transformation, validation and publication steps.

    The skill supports vendor-neutral SQL, Python, Spark, warehouse and orchestration designs, with platform-specific guidance only when context is supplied. It plans deterministic transformations, incremental boundaries, schema changes, late data, backfills, idempotency and source-to-target reconciliation. Correctness comes before performance optimization.

    Works in ChatGPT and Claude with supplied schemas and authorized data. Generated designs and code require representative testing in the exact environment. It does not invent schemas, row counts, connector behavior, performance gains or successful production deployments.

    Behavior depends on exact engines, schemas, connectors, data and transaction semantics. Generated workflows need representative tests and reconciliation before production. Cannot guarantee performance, infer keys or business definitions, or claim completed loads without execution evidence.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 1 month ago

    Permissions

    Read Files
    Write Files

    File Scopes

    Authorized
    task
    inputs
    and
    agreed
    output
    directory
    only

    Optional authorized schema and sanitized sample-data reading; write agreed design or code outputs. No database credentials, live queries, production loads or deployment implied. Limit data to necessary fields and validate against actual engine semantics.

    Listed1 month ago

    What's inside

    Frequently Asked Questions

    Yes. It separates business and quality requirements from implementation, adapting only when a platform and its constraints are supplied.

    Safety requires explicit idempotency and publication design plus testing in the target environment; it is not assumed.

    No. Representative data tests, grain and key validation, and source-to-target reconciliation are required.

    New in Data & Databases