- Home
- Skills
- Trading & Investing
- backtest truth detector
backtest truth detector
Detects curve-fitting, look-ahead bias, and unrealistic assumptions in trading strategy backtests.
$5
Works with the AI Agents you already use
backtest truth detector
Example session with this skill installed
Audit this strategy: 45% CAGR, 3.2 Sharpe, 15 trades over 6 months on SPY. It uses candle closes for fills with zero slippage. Can I trust these numbers?
- Read your context and instructions
- Compiled the backtest truth detector
Verdict: Fail.
Red Flags
- Sample size (15 trades) is statistically insignificant.
- 0 slippage assumes infinite liquidity; recomputing with 5bps slippage reduces Sharpe to 1.1.
- Signal generation at candle close with immediate fill introduces look-ahead bias.
Recommendation: Do not deploy.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Most backtests look great on paper but fail in live markets due to curve-fitting, look-ahead bias, and ignored execution costs. Developers and traders often waste capital on strategies that rely on unrealistic fills or statistically insignificant samples.
What it does
- Detects look-ahead bias and unrealistic execution assumptions like mid-price fills on large orders.
- Audits sample sizes and market regimes to identify weak statistical significance or survival bias.
- Calculates the impact of missing costs including slippage, bid/ask spreads, and borrow fees.
- Analyzes parameter sensitivity to find evidence of over-optimization and fragile performance spikes.
Why this beats prompting it yourself
General LLMs often take backtest data at face value or miss subtle technical flaws like point-in-time constituent errors. This skill applies a systematic forensic framework that forces a recomputation of metrics under stress, ensuring you see the "net-of-everything" reality before deploying code.
Use cases
- Audit a vendor's high-Sharpe strategy before purchasing or licensing.
- Validate a new quantitative model against out-of-sample data and market impact.
- Identify why a live strategy is diverging from its historical performance.
- Stress-test a portfolio for fragility by removing top outlier trades.
Known limitations
Requires detailed trade logs or strategy parameters to provide a high-confidence verdict. It cannot fix a broken strategy, only identify the flaws in its current state.
Requires strategy parameters or trade logs for deep analysis. Does not perform automated data cleaning or fix underlying code bugs.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 months ago
Permissions
No special permissions declared or detected