backtest truth detector

    1

    Detects curve-fitting, look-ahead bias, and unrealistic assumptions in trading strategy backtests.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    backtest truth detector

    Example session with this skill installed

    Audit this strategy: 45% CAGR, 3.2 Sharpe, 15 trades over 6 months on SPY. It uses candle closes for fills with zero slippage. Can I trust these numbers?

    • Read your context and instructions
    • Compiled the backtest truth detector

    Verdict: Fail.

    Red Flags

    • Sample size (15 trades) is statistically insignificant.
    • 0 slippage assumes infinite liquidity; recomputing with 5bps slippage reduces Sharpe to 1.1.
    • Signal generation at candle close with immediate fill introduces look-ahead bias.
      Recommendation: Do not deploy.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Most backtests look great on paper but fail in live markets due to curve-fitting, look-ahead bias, and ignored execution costs. Developers and traders often waste capital on strategies that rely on unrealistic fills or statistically insignificant samples.

    What it does

    • Detects look-ahead bias and unrealistic execution assumptions like mid-price fills on large orders.
    • Audits sample sizes and market regimes to identify weak statistical significance or survival bias.
    • Calculates the impact of missing costs including slippage, bid/ask spreads, and borrow fees.
    • Analyzes parameter sensitivity to find evidence of over-optimization and fragile performance spikes.

    Why this beats prompting it yourself

    General LLMs often take backtest data at face value or miss subtle technical flaws like point-in-time constituent errors. This skill applies a systematic forensic framework that forces a recomputation of metrics under stress, ensuring you see the "net-of-everything" reality before deploying code.

    Use cases

    • Audit a vendor's high-Sharpe strategy before purchasing or licensing.
    • Validate a new quantitative model against out-of-sample data and market impact.
    • Identify why a live strategy is diverging from its historical performance.
    • Stress-test a portfolio for fragility by removing top outlier trades.

    Known limitations

    Requires detailed trade logs or strategy parameters to provide a high-confidence verdict. It cannot fix a broken strategy, only identify the flaws in its current state.

    Requires strategy parameters or trade logs for deep analysis. Does not perform automated data cleaning or fix underlying code bugs.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 3 months ago

    Permissions

    No special permissions declared or detected

    Listed3 months ago

    What's inside

    Frequently Asked Questions

    This skill acts as a forensic auditor for your trading data, specifically identifying technical flaws like look-ahead bias, curve-fitting, and unrealistic execution costs that typically lead to strategy failure in live markets.

    To provide an accurate audit, you need to provide trade logs (CSV or JSON) or specific strategy parameters; the skill then recomputes performance metrics under stressed conditions to find hidden fragilities.

    The skill is compatible with most major AI agent platforms and LLM frameworks that support data analysis and file processing, requiring no complex external API integrations.

    Your purchase includes the full skill logic, a guide on formatting your trade data for the best results, and access to all future version updates as market regime detection methods evolve.

    No, this skill is designed to provide a 'net-of-everything' reality check on existing strategies; while it identifies why a strategy is failing, it does not rewrite the underlying code for you.

    The skill performs a parameter sensitivity analysis, which checks if a strategy's success depends on very specific settings; if a small change in a variable leads to a total collapse in performance, the skill flags it as over-optimized.

    New in Trading & Investing