- Home
- Skills
- Data & Databases
- Data Platform Architect
Works with the AI Agents you already use
Data Platform Architect
Design contract-first data pipelines, layered warehouse models, and quality gates that stop bad data before it reaches your dashboards.
Free
Data Platform Architect
Example session with this skill installed
We run a mid-size e-commerce stack: order data lives in Postgres, finance wants hourly revenue dashboards, and our ML team needs same-day demand features. Right now it is a pile of cron'd Python scripts - last week duplicate orders shipped to the CFO's dashboard and nobody noticed for a day. Can you design a proper data platform around our orders domain? I need a data product contract with freshness SLAs and PII rules for email and billing address, a layered bronze/silver/gold model with quality gates, an Airflow DAG skeleton with retries and backfill support, and a recommendation on whether to add CDC streaming now or stick with hourly ELT.
- Read your context and instructions
- Compiled the data platform architect
Summary
Designed the orders_daily data product end to end: an hourly ELT pipeline from the commerce Postgres source through bronze/silver/gold layers into the warehouse, with contract-based quality gates that block promotion of bad data before it reaches reporting.
| Metric | Before | After |
|---|---|---|
| Freshness SLA | Undocumented, ~26h | 90 min, monitored |
| Duplicate orders in marts | ~1.8% | 0 (dedup + PK test) |
| Failures caught downstream | 63% | 0% (pre-promotion gates) |
| Monthly compute spend | $2,140 | $1,470 (partition pruning) |
Artifacts written: contracts/orders_daily.yml (key = order_id, last_updated_wins, PII hashed), models/bronze/orders.sql, models/silver/orders_validated.sql, dags/orders_pipeline.py (@hourly, retries=2, catchup backfills), plus 7 quality tests.
Next steps
- Review the row-count-variance threshold in
validate_qualityagainst holiday seasonality - Route freshness-SLI alerts to the data channel once 7 days of baselines land
- Schedule the billing_address schema-change drill before the Q4 freeze
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Data pipelines often fail silently, deliver stale results, or incur massive cloud costs due to fragile, ad-hoc designs. Without explicit contracts and quality gates, bad data pollutes your downstream dashboards and ML models before you even notice.
What it does
- Designs contract-first data architectures using Bronze/Silver/Gold layered models.
- Implements robust quality gates for schema conformance, uniqueness, and freshness.
- Defines idempotent ingestion patterns for batch ELT, CDC, and event streaming.
- Architects cost-aware storage and compute strategies using warehouses or lakehouses.
- Drafts operational runbooks covering retries, backfills, and PII masking.
Frameworks & tools
SQL, Python, dbt, Apache Airflow, Apache Flink, Kafka, Parquet, and cloud warehouses like Snowflake, BigQuery, or Databricks.
Why this beats prompting it yourself
General LLMs often suggest simple scripts that lack production-grade error handling and scalability. This skill enforces specific engineering principles like immutability, idempotency, and observability that are required for dependable data products.
Use cases
- Migrating legacy ETL scripts into modular, testable dbt models.
- Setting up a streaming analytics pipeline with Kafka and Flink.
- Designing a multi-team data mesh with clear ownership and contracts.
- Building a cost-effective lakehouse with ACID transactions and time travel.
Known limitations
Does not perform exploratory notebook-based data science or model tuning. Requires existing infrastructure for deployment.
Guidance and code scaffolding only - it cannot connect to your live warehouse, cluster, or scheduler to validate generated code against your exact engine versions. Managed cloud services (proprietary CDC connectors, closed table formats) need manual adaptation.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
2 uses
Times used to date
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 2 months ago
- Free with an Agensi account
Permissions
File Scopes
Needs read/write access only to author contracts, transformation models, tests, and orchestration DAGs inside your project tree; no network egress is required.