More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Honest Strategy Validation for Trading Bots

    1

    The expensive way to learn that is with real money after weeks of tuning.

    Free

    12 usesSecurity scanned
    Honest Strategy Validation for Trading Bots

    Honest Strategy Validation for Trading Bots

    Example session with this skill installed

    I want to build a scalping bot for BTCUSD on my broker (spread is around $25 at BTC ~95k). Idea: EMA9 crosses EMA21 on the 1-minute chart, hold 5 minutes. I attached two weeks of 1m candles. Can you check if this is worth building before I spend time on it?

    • Read your context and instructions
    • Compiled the honest strategy validation
    • Generated the data export

    Validation: EMA9/EMA21 cross, 5-minute hold, BTCUSD 1m, 2025-01-15 to 2025-01-28

    Verdict

    Don't build it as specified. The signal doesn't predict direction even before costs (gross -0.73 bps per trade, 95% CI -1.9 to +0.3). After the $25 spread you lose about 3.2 bps per trade, with every bootstrap resample negative. This is a cheap "no".

    What I tested

    • Data: your file, 20,160 one-minute bars (14 days, no gaps). BTC traded at about $96.5k–$109k, median ~$103.7k.
    • Rule: EMA9 and EMA21 on 1m closes. When EMA9 crosses above EMA21, go long. When it crosses below, go short. Entry is at the

    open of the next bar, so there is no look-ahead. Exit is at the close 5 minutes later. No stop and no target, since you didn't specify any.

    • Cost: $25 round-trip spread, which is about 2.4 bps at these prices (2.6 bps at $95k). I added

    no commission or slippage, so real results would be worse.

    • The rule fired 966 times, about 69 trades per day.

    Gates

    GateResultEvidence
    0 Accountingn/aNo live P&L yet. This is a pre-build check on bars.
    1 Cost vs move0.28–0.30× at 5 min (0.69× at 1 min)Median absolute 5-minute move is 8.7 bps against 2.4–2.6 bps cost. The horizon is playable only with a strong signal: the spread eats about 30% of a typical move. At 1 min it eats 69%, and up to 0.95× around 06 UTC.
    2 Random baselinerandom side, 5-min hold: gross -0.01 bps, net -2.44 bps, win rate 42.5%A coin flip with these exits loses the spread every trade. That is the bar to beat.
    3 Gross before costs-0.73 bps (day-bootstrap 95% CI -1.90 .. +0.26, PF 0.89, win rate 48.4%)No directional information. Even the top of the CI (+0.26) is about 10× smaller than the 2.4 bps cost. Investigation stops here.
    Net (for reference)-3.16 bps/trade (CI -4.31 .. -2.16, p≈1.00, PF 0.59, win rate 40.8%)Slightly worse than random (-2.44).
    4 Pre-registrationnot doneNot needed: the idea failed the cheaper gates.
    5 Holdoutnot openedSame reason.

    In money terms: trading 1 BTC per signal would have averaged

    -$32 per trade, about -$31k over the two weeks. At 0.01 BTC that is about -$0.32 per trade, or about -$22 per day at ~69 trades/day. These are backtest numbers on 14 days, not a forecast. They only show that the costs win.

    Diagnostics (not used to decide)

    • Gross by horizon, same signals: 1m -0.09, 3m -0.44, 5m -0.73, 10m -0.41, 15m -0.87, 30m +0.24, 60m -0.43 bps. Every horizon is inside noise. No hold time shows an edge, so changing the hold time won't rescue the idea.
    • By side (5 min, gross): long -1.30 bps, short -0.16 bps. Both are within noise.
    • Caveat: 14 days is one regime and a short sample (14 independent days for the bootstrap). That cuts both ways, but a gross upper bound of +0.26 bps against a 2.4 bps cost is a very wide gap to close.

    What would have to change

    For a 5-minute hold to work at this spread, the signal would need to average

    more than +2.5 bps gross per trade, consistently. The best this one can plausibly do is about +0.3. Alternatively, the round-trip cost would need to drop well below ~0.3 bps (raw-spread/ECN pricing), and even then this signal shows nothing to harvest.

    What not to do next

    Don't invert it ("it loses, so trade the opposite"). The gross mean is indistinguishable from zero, so the inverse also has no edge and still pays the spread. An inverted rule would also be a new hypothesis that needs new data.

    • Don't grid-search EMA lengths, hold times, or add RSI/MACD filters on these same two weeks. Those indicators come from the same OHLCV. You'd likely find a combination that looks good in-sample by chance, and it would be fitted noise.

    Don't add a tight stop / wide target to "fix R:R". Exits redistribute the outcomes but cannot create direction (see Gate 2).

    Cheapest honest next step

    If you still want to build something on BTCUSD with this broker, move to horizons where the spread is small relative to the move. At 1 hour, cost/move is 0.09×; at 4 hours it is 0.05×; daily is about 0.01×. Then bring a hypothesis based on information that isn't just a 1m price crossover. Before testing it, write the rule down and lock it (pre-registration), split train/holdout by time, and run it through the same gates.


    Files the agent wrote: ema_cross_gross.py (signal + random baseline), gate1_cost_vs_move.txt, gate2_3_ema_cross.txt, gate6_day_bootstrap.txt, trades.csv.

    trades.csv

    CSV · data export

    Generated

    Example file from a real run - the skill writes it into your workspace.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Most trading strategies have no edge. The expensive way to learn that is with real money after weeks of tuning. This skill makes your agent find out first, with the cheapest test that can kill the idea, and only moves to slower tests when the idea survives.

    It turns your agent into a skeptical quant colleague, not a cheerleader. It reports what the numbers say, including "this has no edge", and never presents a backtest as proof of future profit.

    What it checks, cheapest first

    GateQuestion
    0 AccountingIs the P&L real? Broker net vs bot log, including the MT5 trap where closing deals carry magic 0
    1 Cost vs moveCan this holding period pay its round-trip cost at all?
    2 Random baselineWhat do your exact stop, target and time exit produce with a coin-flip entry?
    3 Gross before costsDoes the signal know the direction, before any cost?
    4 Pre-registrationRule, costs and pass bar written and SHA-256 locked before the deciding test
    5 HoldoutTrain, then one look at unseen data, and only once
    6 StatisticsBootstrap by calendar day, so same-day trades don't fake confidence

    Problems it catches

    • "My bot wins 72% but loses money" (a wide stop and tight target win 72% on random entries)
    • "5:1 reward/risk means I can be wrong 80% of the time" (with no entry edge you will be, and break even at best)
    • Adding indicator votes built from the same candles
    • Inverting a losing rule, picking the best horizon after the fact, re-opening a holdout
    • Survivorship (today's symbol list applied to past years)
    • Sizing with an LLM's stated probability
    • A real effect smaller than your spread

    What's inside

    • SKILL.md: the gated workflow and a fixed report format
    • references/pitfalls.md: 10 traps with numbers from a real retail project that tested strategy after strategy on demo accounts
    • references/accounting_traps.md: MetaTrader 5 and exchange P&L traps
    • references/prereg_template.md: pre-registration template
    • 4 scripts (Python + numpy, any CSV of OHLC bars): cost_vs_move.py, random_baseline.py, day_bootstrap.py, prereg_lock.py

    Tested

    Evaluated against the same agent without the skill on 8 realistic cases. With the skill it passed every check (100%). Without it, it missed measuring the rule before costs when asked to invert a losing strategy, and on another case it grid-searched 20 parameter sets on the test data. With the skill the agent also pre-registered and locked the rule, kept a holdout unopened, and traced a "55% win rate" to the market trend rather than the rule.

    It will not hand you a profitable strategy. It tells you, quickly and with evidence, whether the one you have is worth your money.

    It does not generate strategies and does not promise profits; most strategies it checks fail, which is the point. Results are only as good as the data and cost assumptions you give it: spot data standing in for a CFD, or a spread that widens in fast markets, can still mislead. The scripts work on bar data, not tick or order-book data. It is analysis, not financial advice.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    12 uses

    Times used to date

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 19 days ago

    • Free with an Agensi account

    Requires

    Python 3.9+
    numpy

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Network Access

    File Scopes

    *.csv
    PREREG_*.md

    Runs local Python scripts on CSV price files you provide, and writes a .sha256 lock file next to pre-registration files. No network access, no broker connection, no credentials.

    Listed19 days ago
    Updated17 days ago

    What's inside

    Frequently Asked Questions

    This skill acts as a rigorous filter that identifies common backtesting errors—like ignoring swap costs, slippage, or statistical noise—to tell you if a strategy has a genuine edge before you risk capital.

    The skill provides a specialized evaluation framework, a pre-registration template for audit trails, and the specific Python logic needed to perform advanced statistical tests like day-clustered bootstrapping.

    Yes, this skill is market-agnostic and works with any asset class—including crypto, forex, and equities—as long as you can provide OHLCV or trade log data for analysis.

    This skill is designed to work with most advanced LLMs (like GPT-4 or Claude 3.5) that support Python code execution or can interpret data analysis results from the provided scripts.

    No, this is a diagnostic tool designed to 'kill' bad ideas; it requires you to provide the trade data or strategy rules, which it then subjects to rigorous quant-grade stress tests.

    While a basic understanding of Python helps run the validation scripts, the skill's primary value is in the 'skeptical quant' logic it uses to interpret your data and guide your decisions.

    No. It tests the strategy you already have and tells you whether it has an edge after costs. Most don't; finding that out before risking money is the value.

    No. It works with any market and broker. MT5 gets extra coverage because its P&L accounting has traps that are easy to miss.

    A CSV of bars with time, open, high, low and close. A spread column is optional. For P&L questions, your broker's deal history.

    No. Everything runs locally on files you provide.

    New in Trading & Investing