- Home
- Skills
- Data & Databases
- Data Storage and Architecture
More screenshots
Works with the AI Agents you already use
Data Storage and Architecture
Model warehouses, lakehouses, Iceberg and data mesh. 10 ordered skills, starting with Dimensional Modeling, Modern Data Warehouse, Data Lakehouse.
Free
Data Storage and Architecture
Example session with this skill installed
We have a slow query on our Iceberg table in Snowflake. Users filter by event_timestamp. Can you review our partition spec and suggest how to evolve it without a full rewrite?
- Read your context and instructions
- Compiled the data storage
I have analyzed the partitioning requirements for your Iceberg table. Since users frequently filter by event_timestamp, the current identity partitioning is likely creating too many small files or scanning too much data if the granularity is off.
Iceberg Partition Evolution Plan
| Component | Current State | Proposed Change | Rationale |
|---|---|---|---|
| Partition Transform | identity(event_timestamp) | days(event_timestamp) | Reduces manifest size and metadata overhead for daily queries. |
| Evolution Method | Manual Rewrite | ALTER TABLE ... SET PARTITION SPEC | Iceberg supports in-place evolution; old data stays in old layout, new data uses new layout. |
| File Compaction | Fragmented small files | rewrite_data_files | Use the Spark or Trino procedure to bin-pack existing small files into larger Parquet files. |
| Format Version | v1 | v2 | Required if you plan to use row-level deletes (merge/update) efficiently with the new spec. |
-- Step 1: Evolve the partition spec (Metadata only change)
ALTER TABLE events_table SET PARTITION SPEC (
days(event_timestamp)
);
-- Step 2: Optimize existing data to the new layout (Optional but recommended)
-- This prevents the "split-brain" performance where old data is still slow
CALL system.rewrite_data_files(
table => 'events_table',
options => map('target-file-size-bytes', '536870912') -- 512MB target
);
Next steps
- Run
EXPLAINon a typical query to see if the optimizer is currently hittingManifestListbottlenecks. - Confirm your engine version (Snowflake/Spark/Trino) supports Iceberg Spec v2 if enabling row-level deletes.
- Schedule a snapshot expiration job to clean up metadata files after the rewrite.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Data architects often need a repeatable process, not a one-off prompt. This pack targets: Model warehouses, lakehouses, Iceberg and data mesh.
What's inside (10 skills, suggested learning order)
- Dimensional Modeling — Star schemas for analytics.
- Modern Data Warehouse — Design cloud warehouses.
- Data Lakehouse — Unify lake and warehouse.
- Apache Iceberg — Open table format.
- Data Mesh — Decentralized data architecture.
- Relational Data Modeling — Design database schemas.
- SQL Query Optimization — Improve SQL performance.
- Data Warehouse Automation — Automate modeling and loading.
- Enterprise Data Architecture — Design data platforms.
- DataOps Operations — Apply DevOps practices to data.
Why a skill pack instead of a one-off prompt
Each skill gives the agent a defined trigger, step-by-step procedure and output format, so results are consistent across sessions and easier to review.
How to use
- Install the pack in your SKILL.md-compatible agent.
- Ask for the task in plain language.
- Review the output against the skill's quality checks before using it.
Known limitations
- Outputs are drafts that need human review; they are not professional, legal or financial advice.
- Tools, APIs and prices change; check current vendor documentation before relying on version-specific details.
- Quality depends on the context you give the agent (data, requirements, constraints).
- Tested only where stated in the Compatibility note.
Also by Rodrigo Santos
Specialized Agents and Distribution; AI Agents: Fundamentals and Protocols; Model Fine-Tuning and Alignment
Does not tune non-relational DBs. Architecture focus is limited to data domains (lakehouse, mesh, warehouse). Requires engine versions for specific feature support.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 3 days ago
- Free with an Agensi account