The Vault Test: A Near-Blind Out-of-Sample Validation of CoinRoc’s Three-State FIS, 2023–2025

What Makes This a Near-Blind Study

CoinRoc’s three-state Fuzzy Inference System (FIS) regime thresholds, grid spacing parameters, and momentum sleeve settings were locked on training data through 2022. Then, without adjustment, we ran the strategy on the 2023, 2024, and 2025 test years. No parameter was touched after seeing any 2023–2025 price data. This study reports what happened across 51 out-of-sample folds.

What makes this a near-blind out-of-sample study: The walk-forward design ensures every test year’s strategy behavior is driven by training data that predates the test period — no FIS thresholds, grid spacing parameters, or sleeve sensitivity settings were adjusted based on 2023–2025 observations. A residual limitation applies: the parabolic detector’s membership function boundaries were calibrated using 2021 price history as the design case. The study is stronger than an in-sample result (test data is genuinely out-of-sample) but weaker than a theoretically pure blind design that had no knowledge of 2021 parabolic events. This distinction is material and is disclosed in full in the Limitations section.

Why “Out-of-Sample” Matters More Than “Backtested”

Every grid bot platform shows a backtest. The Binance Grid Bot has a configuration screen where you can select a date range, run the simulation on historical price data, and see an outcome. So does 3Commas. So does Pionex.

What those backtests share: the person configuring them already knows what happened. You can see the chart, pick a sideways period, and set your grid bounds to exactly capture the range. The resulting return figure looks compelling. It was built from knowledge you had at configuration time.

The walk-forward methodology used in CoinRoc’s research papers is different in one specific way: the algorithm is not allowed to see the test year’s data during training. For every fold in this study:

• Test year 2023: strategy behavior determined entirely by 2021–2022 training data
• Test year 2024: strategy behavior determined entirely by 2022–2023 training data
• Test year 2025: strategy behavior determined entirely by 2023–2024 training data

Zero parameters were adjusted after observing 2023–2025 price action. The thresholds, spacing settings, and sleeve allocations that governed strategy behavior in 2023 were the same ones set during the 2019–2022 design phase. That is the vault: parameters locked, sealed, and run forward without touching.

Methodology

2.1 Study Structure

2.2 Three Variants Compared

2.3 Cost Model

All returns are simulated gross under a retail Binance.US cost structure: 0.40% maker / 0.60% taker / 0.05% slippage, with grid reset repositioning costs modeled. Net-of-fees returns may differ.

2.4 Proxy Methodology Note

The ParabolicRegimeDetector has not yet been run bar-by-bar against the 2023–2025 price paths (that step is pending implementation). This study uses a tier-based proxy validated against Gate 1 findings. Annual fold buy-and-hold return serves as a proxy for intra-year velocity concentration. The proxy establishes the directional case; quantitative precision requires bar-by-bar simulation, which is the next milestone.

The 51 Out-of-Sample Folds

All strategy parameters fixed on pre-2023 training data.

Summary Statistics — 2023–2025 Out-of-Sample Window

4.1 Strategy Return Comparison — All 51 Folds

All figures are simulated/backtested under the assumptions described in Section 2.

Note on B&H mean: The B&H mean of +46.2% is heavily inflated by STXUSD 2023 (+605%), ZECUSD 2025 (+807%), and DOGEUSD 2024 (+242%). The median B&H of +11.2% is more representative of the typical fold experience.

Note on median strategy return: The 0.0% median is a structural feature of the strategy, not a failure mode. 26 of 51 folds are gated-zero outcomes — the strategy held cash during bear and trending regimes. The 25 folds with positive returns carry the mean.

Three-Strategy Comparison, Year by Year

4.2 Bear-Fold Detail (20 folds)

Across all 20 bear folds in the 2023–2025 out-of-sample window, the strategy produced a higher return than buy-and-hold in every instance (20/20, 100%) — a result derived from backtested/simulated data under the assumptions described in Section 2. Simulated past performance does not guarantee future results.

16 of 20 bear folds were FIS-gated: strategy earned 0% vs. B&H ranging from −21.0% to −84.1%. Four bear folds had the grid running (random-walk regime during a bear year): DOGEUSD 2025 (−9.3% vs −63.9%), LTCUSD 2025 (+3.8% vs −26.9%), XLMUSD 2025 (+0.8% vs −52.8%), XRPUSD 2025 (+1.6% vs −21.0%). All four outperformed buy-and-hold.

4.3 Parabolic Events in the Out-of-Sample Window

‡ ZECUSD 2025 partial-year caveat applies throughout. See Limitations.

The two parabolic-tier folds (STXUSD 2023, ZECUSD 2025) contribute +141.2% total sleeve contribution across 51 folds combined. Excluding both, mean three-state return drops from +3.8% to +1.9%. In a year with no parabolic event, the three-state strategy performs identically to the two-state strategy in expectation. The sleeve adds value specifically in parabolic-year scenarios.

Two-State vs. Three-State — Direct Comparison

All figures are simulated/backtested. ‡ ZECUSD 2025 is an ongoing fold — partial-year data only. See Limitations.

Does the Three-State FIS Generate Statistically Superior Returns?

51 folds is not 51 independent observations. Cross-asset correlation in crypto is high (typically 0.5–0.8 in bull cycles), which reduces the effective independent observation count to roughly 12–18 after accounting for co-movement. The three-state return distribution has heavy right-tail skew — STXUSD 2023 (+60.5%) and ZECUSD 2025 (+80.7%) dominate the mean. Standard t-tests fail normality assumptions; block-bootstrap confidence intervals are the appropriate method and require bar-by-bar simulation data not yet available from the proxy.

The appropriate framing: because the directional improvement arises from the structural behavior of the direction guard (which prevents the sleeve from firing on bear folds by design), the case for the three-state strategy rests on the logic of the mechanism, not on the magnitude of the return difference alone. The return improvement is a consequence of the mechanism operating correctly, and it remains subject to the quantification uncertainty described in the Limitations section.

What This Strategy Does That Generic Grid Bots Cannot

6.1 The Trap Problem

Generic grid bots — available on Binance, 3Commas, Pionex, Bitsgap, and dozens of exchanges — run with static upper and lower bounds set at configuration time. No regime awareness. No adjustment based on market conditions. In a strong downtrend, the bot continues placing buy orders as price falls, accumulating directional exposure at each level. By the time price exits the grid range downward, the account is fully deployed in a position that has been averaging down the entire way. This is the trap.

The FIS gate is the solution. When the Hurst exponent on CoinRoc’s training window rises above 0.55 — indicating a persistent trend rather than mean-reverting oscillation — the grid pauses and holds cash. The strategy does not place orders into the trend. This is the single feature that separates the strategy from the trap outcome above.

6.2 The Illustrative Dollar Case — ETH 2022

* The "always-on grid bot" outcome (~$3,500–$4,500) is an illustrative estimate based on the known ETH 2022 B&H return of -67.5% and a model assumption that a static grid partially fills into the drawdown, accumulating directional exposure at roughly 50–65% of the B&H loss. This figure is not sourced from a measured Binance Grid Bot or 3Commas backtest result. Actual competitor outcomes in 2022 may differ materially from this estimate. The CoinRoc ETHUSD 2022 result (0%, cash) is directly measured from the walk-forward study. See Methodology and Limitations.

6.3 Research Validation Axis

Generic grid bot backtests, where they exist, are typically: single path (no confidence intervals), user-selected time window, no transaction costs modeled accurately, survivorship-biased, no out-of-sample validation. The same window used to configure the bot is used to show the backtest.

CoinRoc’s validation methodology: 85-fold walk-forward for the modern era (2019–2025), 2-year train / 1-year test rolling window. Each test year’s strategy behavior is driven only by the preceding training window. Full cost model. 17-asset universe. Bear-fold analysis explicitly reported. This paper: near-blind 2023–2025 validation where strategy parameters were fully fixed before the test window opened.

As of the date of this study, no publicly documented competing grid bot platform has disclosed a regime-adaptive momentum capture layer of comparable architecture. The three-state FIS classifies markets into three states — ranging (grid), trending non-parabolic (cash), parabolic (trailing-stop momentum sleeve) — and manages capital separately across each state. The Mamdani rule base for the parabolic detector uses an 8-rule fuzzy system producing a continuous parabolicScore output. Every firing decision produces a natural-language explanation traceable to rule activations. This explainability is a technical property of the Mamdani architecture.

6.4 Competitive Positioning Matrix

Limitations

Disclosures

Yodacom Research — 2026-06-09 · Companion to Paper 8: grid-modern-era-2026 · Han Kessel (research), Lando (editorial)