Metrics

Every backtest Forven runs produces a block of metrics describing how a strategy behaved on historical data. This page explains each metric, the flags that mark a metric untrustworthy, and the one distinction that matters most: in-sample versus out-of-sample.

The short version: a strategy is split into a training window and an unseen window. Metrics computed on the unseen window are the ones the promotion gates read, and the ones you should read too.

Forven is a research tool. The numbers on these pages describe past behaviour on historical data. They are illustrative, they are not predictive, and nothing here is financial advice.

In-sample vs out-of-sample

Forven splits every backtest's data 70/30 at a fixed index:

• In-sample (IS) — the first 70% of the data. Signals are generated and tuned here. IS Sharpe is optimistic and does not predict live behaviour.
• Out-of-sample (OOS) — the last 30%, unseen during signal generation. OOS metrics are the ground truth for fitness gates and any deployment decision.

The lab renders IS and OOS side by side. A strategy that looks excellent IS and poor OOS is overfit. A strategy that holds up across both is worth promoting to paper (the public site calls this a "candidate").

A backtest result returns three blocks: in_sample, out_of_sample, and robustness. When you read a result, start with out_of_sample.

The core metrics

compute_metrics() derives 30+ values from each trade ledger. The ones you will read most often:

These are computed identically for the IS and OOS blocks, so you can compare the same strategy across the train/test split directly.

How Sharpe is computed

Sharpe and Sortino are computed from the per-bar returns of the mark-to-market equity curve — every bar counts, including bars where the strategy was flat — and annualized by bars per year. That is a calendar measure: a strategy that trades rarely is not flattered by averaging only its few trades. The older per-trade figure is still reported as trade_sharpe / trade_sortino for comparison. (The paper→live gate's paper Sharpe is a different thing again: a per-trade t-statistic over forward paper trades — see promotion gates.)

Deflated Sharpe

Strategies are optimized before they are validated, and the best of many tries always looks better than it is. The deflated Sharpe ratio estimates the probability that a Sharpe is genuine given how many variants were tried — including optimizer trials and disproven sibling ideas from the same family and asset. Forven always computes it and shows the last value in The Forge; the gate that rejects on it is opt-in.

Breakdowns

Two slices help you see where performance came from:

• by_side — long vs short. A strategy that only makes money on one side is really half a strategy.
• by_regime — performance split across the four market regimes (trend_up, trend_down, range_bound, high_vol). This shows whether an edge is regime-specific or broad-based.

Robustness and walk-forward

A single 70/30 split is one test. The gauntlet goes further with walk-forward analysis (WFA) — N-fold cross-validation that re-runs the IS/OOS split across multiple windows.

Two robustness figures matter:

• Robustness score — defined as 1.0 - max(IS→OOS Sharpe degradation, 0). A high score means performance held up from training to test; a low score means the edge decayed out-of-sample.
• Degradation % — 1 - (avg_oos_sharpe / avg_is_sharpe) aggregated across folds. A standalone walk-forward run reports PASS when degradation is below 50% and there are at least 5 OOS trades. The gauntlet then holds it to a stricter bar — degradation of at most 35%, enough out-of-sample trades, at least 2 judgeable folds, and a minimum share of profitable folds — see the gauntlet.

Defaults: 5 folds, 70% IS / 30% OOS per fold (walkforward_folds, walkforward_train_ratio). WFA needs at least 420 bars of data, and the gauntlet sizes the window from the strategy's own trade rate so each fold has enough trades to judge. Adequacy warnings are logged when an OOS window is shorter than 30 days or a fold has too few bars — the numbers still compute, but treat them as weak.

Reliability flags — read these first

Short windows produce metrics that look spectacular and mean nothing. Forven attaches boolean flags so the gates and the UI can suppress numbers that aren't yet trustworthy.

Two reasons these exist:

1. Sharpe annualization inflates wildly on short samples. Under 20 trades, the figure is flagged unreliable.
2. Annualized return is capped at a 3-month window for display. A 25-day run can report a return in the thousands of percent — that is an artifact, not a result. The flag suppresses the display; it does not change the underlying fitness math.

When a flag is false, the number is hidden rather than shown with a false air of confidence. That is deliberate: transparency over confidence.

Funding-cost provenance

Forven applies perpetual-futures funding to backtest trades by default (backtest_include_funding), using Binance funding prints normalized per hour by each print's own interval and accrued inside the execution kernel exactly once. Because a funding-blind backtest can look better than reality, two flags record what actually happened:

• funding_applied — at least one trade carried a funding cost.
• funding_complete — every trade had complete funding history.

If funding_complete is false, the promotion gates may reject the run rather than promote a strategy validated in a window with missing funding data. On a fresh install the first backtest auto-backfills funding history from the exchange, so this self-heals over time.

How fees, slippage, and leverage enter the numbers

Metrics already include trading costs. The defaults applied to each round-trip:

Fees and slippage are charged on entry and on exit. A strategy's metrics therefore reflect a return after costs — there is no separate "gross vs net" toggle to forget. In risk-based sizing, leverage does not change the loss at the stop: the position shrinks as leverage rises.

Metric semantics are versioned

Every verdict is stamped with the backtest engine version and a fingerprint of the data it was scored on. When the way a metric is computed changes — the move to per-bar Sharpe was one such change — older verdicts are marked stale and re-validated automatically, rather than silently compared against new ones. See promotion gates.

How the gates read metrics

The promotion gates compute a single fitness score (0–100) from five weighted factors, all read from the OOS metrics:

fitness = 30% Sharpe (capped at 3.0)
        + 20% win rate
        + 20% profit factor (capped at 5.0)
        + 15% max drawdown (penalty: 0 at 10%+ drawdown)
        + 15% trade-count bonus (min 20 trades)

The thresholds that move a strategy along the pipeline:

• >= 60 — eligible for paper (the public "candidate" stage)
• >= 70 — eligible for deploy / live
• < 40 — eligible for retirement

A metrics integrity check (check_metrics_integrity()) runs before any promotion. Impossible ratios, NaN, and other anomalies quarantine the result and block promotion — a strategy cannot be promoted on numbers that don't add up.

Reading a result via the API

You can pull the raw metrics directly. Submit a run and read the result:

# Submit a backtest
curl.exe -X POST http://127.0.0.1:8003/api/backtesting/run `
  -H "x-api-key: $env:FORVEN_API_KEY" `
  -H "content-type: application/json" `
  -d '{ "strategy_type": "rsi_momentum", "asset": "BTC", "timeframe": "1h" }'

# Retrieve the full ledger + metrics
curl.exe http://127.0.0.1:8003/api/results/<result_id> `
  -H "x-api-key: $env:FORVEN_API_KEY"

The response carries the in_sample, out_of_sample, and robustness blocks described above. In the UI, the same data renders in a backtest's full report — chart, trades table, and metrics panel, with regime shadings on the chart — and in All Backtests (/lab/backtests).

Caveats

• Trust OOS, not IS. In-sample metrics are tuning artifacts. The gates only read out-of-sample, and so should you.
• Honor the flags. A great Sharpe under 20 trades or a four-digit annualized return on a three-week window is noise, not edge.
• Infinite profit factor is real. A strategy with zero losses reports profit_factor = inf plus profit_factor_is_infinite. It usually means too few trades, not a perfect strategy.
• Costs are baked in. Fees, slippage, and funding are already in the metrics; don't double-count them.
• Past is not prologue. Every number here describes historical behaviour on historical data. It is illustrative, not predictive, and nothing here is financial advice.

Related

• The gauntlet — the full robustness battery these metrics feed
• Backtesting a strategy — where these metrics are produced
• Promotion gates — how fitness and reliability flags gate the pipeline
• Market regimes — the regimes behind the by_regime breakdown