Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    leakage aware backtest auditor

    1

    Audit backtests for data leakage, verify point-in-time feature availability, and calculate cost-adjusted returns.

    $18

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    leakage aware backtest auditor

    leakage aware backtest auditor

    Example session with this skill installed

    Audit this backtest JSON for the period 2020-2024. Check for leakage in the sentiment feature and apply a 5bps cost deduction per rebalance. Data: input_2024_strat.json

    • Read your context and instructions
    • Compiled the leakage aware backtest

    Audit complete for strategy ID: STRAT-2024-001. I have processed the return series and verified feature availability timestamps against the provided data snapshot. Leakage was detected in the 'Sentiment_Score' feature during the 2021-Q2 window due to revision vintages appearing before publication dates.

    Backtest Audit Report

    MetricValueStatusNotes
    Audit Window2020-01-01 to 2024-01-01PASSChronological separation maintained.
    Leakage CheckSentiment_Score (2021-Q2)FAILPoint-in-time timestamp mismatch.
    Gross Return42.15%-Cumulative across period.
    Net Return36.82%-Adjusted for 5bps cost/rebalance.
    Max Drawdown-12.4%-Net compounded basis.
    Survivorship BiasIndeterminateWARNMissing delisting history in universe.

    Analysis

    The strategy shows high sensitivity to rebalance frequency. The 5bps cost deduction reduced the Sharpe ratio from 1.8 to 1.4. The leakage in the sentiment feature (v2.1) suggests the backtest used revised data not available until 48 hours post-event.

    Next steps

    1. Re-run the backtest using the Sentiment_Score_Lagged field to resolve the 2021-Q2 leakage.
    2. Provide universe membership history to clear the survivorship bias warning.
    3. Invoke the helper with the --output report.json flag to save the scored series for further walk-forward analysis.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Standard backtests often suffer from look-ahead bias and unrealistic cost assumptions. Most tools fail to account for the actual publication time of data, leading to inflated performance that vanishes in live trading.

    What it does

    • Audits feature availability by mapping every data point to its earliest possible publication timestamp.
    • Enforces strict chronological separation between training and test sets to prevent data leakage.
    • Calculates cost-adjusted returns by applying fractional deductions to period-specific performance.
    • Generates sensitivity tables across independent walk-forward windows and parameter variants.
    • Identifies missing evidence of survivorship bias and corporate-action handling.

    Frameworks & tools

    Python 3 for execution, JSON for data contracts, and local CLI scripts for analysis.

    Why this beats prompting it yourself

    General LLMs struggle with the rigorous, multi-step verification required to prove train/test isolation. This skill follows a specific audit protocol that treats source references as metadata rather than proof, forcing an evidence-based evaluation of strategy robustness.

    Use cases

    • Auditing a new quantitative strategy before committing capital.
    • Identifying look-ahead bias in third-party backtest reports.
    • Standardizing performance reporting across multiple walk-forward windows.
    • Stress-testing return series against varying slippage and commission tiers.

    Known limitations

    Does not execute raw strategy code or fetch market data. Requires pre-computed return series and data availability timestamps as input.

    Requires pre-formatted JSON input following a specific contract. Cannot independently verify data provenance or execute live trades. Limited to auditing supplied return series.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 1 month ago

    Requires

    Python

    Permissions

    Terminal / Shell
    Listed1 month ago

    What's inside

    Frequently Asked Questions

    It checks supplied feature-availability timestamps and train/test chronology, then calculates net returns after supplied period costs. It does not verify publication records, fetch data, or distinguish slippage from commissions automatically.

    Provide normalized JSON with feature availability and decision timestamps, strictly ordered train/test periods, gross returns, nonnegative period costs, and a supplied universe-evidence reference.

    It is designed for AI agents with Python execution capabilities and can be integrated into any workflow that supports JSON data contracts and local CLI scripts.

    No, this skill is strictly an evaluation tool; it requires your existing results as input and does not execute strategy logic or connect to live market data feeds.

    The package includes the instructions, local Python audit helper, JSON input contract, synthetic valid and adverse fixtures, example report, runtime notes, smoke-test script, source notes, and logo. It does not generate sensitivity tables or run backtests.

    It flags features unavailable at the decision time and detects training periods that appear after the test boundary. It does not enforce data isolation or independently verify data provenance.

    New in Trading & Investing