Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    data lineage and provenance mapper

    1

    Data pipelines often become black boxes where the origin of a specific metric is lost across multiple transformations.

    $2.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    data lineage and provenance mapper

    data lineage and provenance mapper

    Example session with this skill installed

    Trace the 'Monthly_Churn_Rate' metric from the Retention_Report back to its sources. I have provided the SQL transformation for the 'Calculated_Churn' node and the source table schemas.

    • Read your context and instructions
    • Compiled the data lineage

    I have processed the lineage for the Monthly_Churn_Rate metric based on the provided pipeline metadata. The deterministic trace is complete.

    Data Lineage and Provenance Report — Analytics Dept, 2024-05-20

    1. Scope, access and sources

    • Sources: E1 (Customer_Subscription_Logs), E2 (Billing_Records)
    • Access Scope: Internal
    • Nodes processed: 3 (2 sources, 1 transform)
    • Metrics excluded: 0
    • Assumptions: user_supplied (transformation logic)

    2. Lineage graph

    idkindclassificationownerapprovedcolumns
    E1sourceinternalDataOpstruesub_id, status, timestamp
    E2sourceinternalFinancetruesub_id, amount, date
    T1transforminternalAnalyticstruechurn_count, total_base

    Edges for Monthly_Churn_Rate:

    • Monthly_Churn_Rate -> T1.churn_count (Direct)
    • T1.churn_count -> E1.status (Conditional: status='cancelled')

    3. Provenance ledger

    MetricReportTargetStatusEvidence gradeSource columnsLongest path
    Monthly_Churn_RateRetention_ReportDashboard_V2tracedvalidatedE1.status, E1.sub_id2 hops

    4. Untraceable and blocked measures

    None.

    5. Findings and items for human review

    • Finding INF-01: Information gap — organizational validation required for E2 billing alignment.
    • Warning: Transformation uses a hardcoded constant for 'Active' status definitions.

    6. Impact analysis

    • Impact Request: Change to E1.status
    • Downstream: T1.churn_count (Dist: 1), Monthly_Churn_Rate (Dist: 2).
    • Owners: Analytics Team, Retention Lead.

    7. Proposed actions (drafts), assumptions and limitations

    • Action: Approve Monthly_Churn_Rate for executive review (status: draft).
    • Observation: Computed result shows 100% dependency on E1 subscription logs.

    This analysis supports, and does not replace, professional judgment and management's decision.

    Management decision required for final publication.

    Next steps

    1. Review the hardcoded constants flagged in section 5.
    2. Confirm the E2 billing record owner approves the joined logic.
    3. Validate the churn_count expression against the latest business requirements.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Data pipelines often become black boxes where the origin of a specific metric is lost across multiple transformations. This skill reconstructs column-level lineage and provenance using your pipeline metadata, ensuring every published figure is traceable to an approved source.

    What it does

    • Lineage Reconstruction traces metrics from reports back through transformations to original source columns.
    • Provenance Mapping generates a ledger detailing source approval status, validation grades, and fallback rules.
    • Integrity Flagging identifies untraceable measures or metrics drawing from unverified and unapproved sources.
    • Impact Analysis predicts downstream effects on reports and metrics when a source column or table is modified.
    • Access Verification checks that metrics do not expose sensitive or confidential columns beyond the defined access scope.

    How it works

    1. Metadata Collection involves registering sources with unique identifiers (E-numbers) and defining node transformations.
    2. Request Formatting builds a request.json file following a strict schema including nodes, columns, and access scopes.
    3. Deterministic Parsing runs a specialized Python script to build the dependency graph and trace each metric logic.
    4. Report Generation produces a structured provenance report with findings, severity codes, and items for human review.

    Frameworks & tools

    The skill utilizes a self-contained Python execution environment to process JSON-based metadata schemas. It operates entirely on supplied metadata without external database connections.

    Why this beats prompting it yourself

    Manual tracing of nested SQL expressions and cross-table dependencies is prone to hallucination in standard LLMs. This skill uses a deterministic script to ensure lineage is calculated rather than guessed, providing exact evidence grades and source references.

    Use cases

    • Auditing a financial report to verify that all figures originate from approved ERP tables.
    • Performing a change impact assessment before refactoring a core data warehouse table.
    • Identifying "shadow data" metrics that rely on unvalidated or estimated inputs.
    • Mapping data sensitivity flows to ensure PII is not leaked into public-facing dashboards.

    Known limitations

    This skill only analyzes provided metadata and does not connect to live databases or file systems. It requires explicit transformation logic or column dependencies to be defined in the input.

    Requires manual supply of metadata via JSON. No live system connectivity. Does not verify if the actual data matches the metadata. Interpretation is restricted to the provided access scope.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 2 days ago

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Environment Variables
    Listed2 days ago

    What's inside

    Frequently Asked Questions

    This skill uses a deterministic Python-based dependency engine rather than relying on LLM guesswork, ensuring your lineage graphs are based on hard logic rather than probabilistic 'hallucinations' common in standard prompting.

    The skill expects a specific JSON-based metadata schema (request.json) containing your E-number identifiers, node transformations, and column dependencies. You do not need to provide access to your live database; it operates entirely on the metadata you supply.

    This skill is designed for any AI agent capable of running Python scripts and handling structured JSON data. It is highly portable and works within self-contained execution environments that support standard Python libraries.

    The purchase includes the core mapping script, the metadata schema definitions, and a comprehensive guide on how to format your pipeline data for optimal results. You also receive template reports for lineage findings and impact analysis.

    No, this skill is a 'zero-connect' tool that analyzes provided metadata offline. This makes it ideal for highly secure environments where the AI agent cannot be granted direct access to production databases or file systems.

    Yes, the skill includes a 'Change Impact' mode that predicts which downstream reports, metrics, and dashboards will be affected if a specific source table or column is modified or deprecated.