Walk-Forward Toolkit

    1

    Audit trading strategies for overfitting using rigorous walk-forward optimization and cost-sensitivity analysis.

    Free

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Walk-Forward Toolkit

    Example session with this skill installed

    Validate this SMA crossover strategy on BTC/USDT daily data. Parameters: fast [5, 10, 20], slow [50, 100, 200]. Use a 2-year train window and 6-month test window with a 50-bar embargo.

    • Read your context and instructions
    • Compiled the walk-forward toolkit

    I have completed the walk-forward audit for the SMA crossover strategy. The analysis shows significant performance degradation between in-sample training and out-of-sample testing windows.

    Walk-Forward Audit Results
    Metric In-Sample (Mean) Out-of-Sample (Mean) WFO Efficiency
    Annualized Return 24.5% 8.2% 0.33
    Profit Factor 1.85 1.12 0.61
    Max Drawdown -12.4% -19.8% N/A (OOS is 1.6x deeper)
    Num. Trades 142 34 N/A
    Parameter Stability & Sensitivity
    Consensus Parameters: fast: 10, slow: 100 (selected in 3 of 5 windows).
    Plateau Score: 0.42 (the chosen parameters sit on a narrow peak rather than a robust plateau).
    Cost Sensitivity: at 2x assumed costs the Profit Factor falls to 0.94, below break-even.
    Threshold Scaling Check
    Status: N/A (strategy uses relative moving averages, no absolute thresholds detected).
    Verdict: FAIL

    The strategy fails the audit because the Walk-Forward Efficiency on annualized return is 0.33, well below the 0.50 threshold. The Profit Factor degradation (1.85 to 1.12) and the shallow plateau score reinforce the reading: most of the in-sample edge is an artifact of parameter optimization rather than a persistent market signal. The strategy is also fragile to increased trading costs, which is a separate warning sign.

    Next steps

    Before iterating, compare the OOS performance against a passive buy-and-hold on the same bars. If the passive benchmark matches or beats the filtered strategy, the filters are adding no edge over simply being in the market.
    Re-run the audit in “anchored” mode (growing training window) to check whether the degradation is due to the rolling window size rather than a genuine lack of signal.
    If the strategy still fails, note that a wider parameter grid will not help and may worsen overfitting. The bottleneck is likely the signal itself, not its parameters.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Overfit trading strategies are the leading cause of capital loss in quantitative finance. Backtests that look spectacular in a notebook often collapse in live markets because they capture noise instead of signal. This skill provides a rigorous audit framework to determine if a strategy's edge survives outside its optimization window.

    What it does

    • Walk-forward optimization Splits data into strictly separated training and testing windows to measure out-of-sample performance.
    • Efficiency scoring Calculates the ratio of out-of-sample returns to in-sample expectations to quantify performance degradation.
    • Parameter stability analysis Tracks if the optimizer selects consistent parameters across windows or chases random noise.
    • Cost sensitivity stress-testing Evaluates Profit Factor survival when trading costs are increased by 1.5x and 2x.
    • Threshold scaling detection Identifies absolute price filters that fail silently as the underlying asset price changes level.

    How it works

    1. Prepare strategy The user defines a Python function that accepts OHLC data and parameters, returning a position series.
    2. Configure Splitter Set window sizes for training, testing, and the mandatory embargo period to prevent data leakage.
    3. Run Engine Execute the walk-forward process across the historical dataset to generate fold-by-fold metrics.
    4. Analyze Verdict Review the automated pass/fail report based on efficiency scores and statistical significance.

    Frameworks & tools

    Built for Python 3.10+ environments using pandas and numpy. It operates on standard OHLC DataFrames and requires no external brokerage connectivity.

    Why this beats prompting it yourself

    Prompting an LLM to "check for overfitting" usually results in generic advice. This skill implements a mathematically sound engine that enforces execution shifts, mandatory embargoes, and cost-first accounting. It replaces vague intuition with an actionable audit trail and numerical proof of an edge.

    Use cases

    • Validating a new strategy before deploying capital to a live brokerage account.
    • Auditing strategies found in academic papers or public repositories to verify claims.
    • Detecting when a previously successful strategy is failing due to parameter drift.
    • Stress-testing a strategy against realistic slippage and commission models.

    Known limitations

    Requires at least 3 years of daily OHLC data for statistical relevance. It is not suitable for high-frequency strategies like market making or non-sequential instruments like options.

    Requires Python 3.10+, pandas, and numpy. Minimum 3 years of daily OHLC data recommended. Not for high-frequency or non-sequential trading strategies. No built-in data downloading.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed today

    • Free with an Agensi account

    Requires

    Python
    Git

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Listedtoday

    What's inside

    Frequently Asked Questions

    This toolkit uses a rolling 'walk-forward' method that tests strategy performance on data the model hasn't seen yet, ensuring your backtest results reflect potential live performance rather than historical coincidences.

    The toolkit is designed for Python 3.10+ environments and works seamlessly with standard pandas OHLC DataFrames, making it compatible with most data-heavy AI agents and quantitative workflows.

    You will receive the core optimization engine, a configuration module for window splitting, and the automated reporting framework for pass/fail strategy audits.

    To ensure statistical significance and enough 'folds' for the walk-forward process, we recommend a minimum of 3 years of daily OHLC data, though more granular data works as long as the historical span is sufficient.

    While it handles slippage and commission scaling, it is designed for sequential trend-following or mean-reversion strategies and is not optimized for high-frequency market making or complex multi-leg options.

    New in Trading & Investing