The research daemon

The research daemon is Forven's autonomous background loop. While Forven is running, it invents new strategy hypotheses, drives candidates through the gauntlet, retires the ones that fail, and surfaces a shortlist of survivors for you to review. It is the engine behind hands-off research.

This is an operator surface. The daemon does real work — spawning hypotheses, running backtests, promoting and archiving strategies — so it is governed by autonomy modes, the scheduler, and operator approval gates. Nothing it produces reaches real capital without surviving the full pipeline and a human approval.

Forven is a research tool. The daemon evaluates strategies against historical and live data; survival in the gauntlet is not a prediction of future results, and nothing here is financial advice.

What the daemon is

"Daemon" covers two cooperating things in Forven. Keep them distinct:

• The research loop — the autonomous discovery-and-testing engine that this page is about. It is a set of scheduled jobs (crucible discovery, the planner, the evolution testing step, paper graduation, learning, and maintenance) that move strategies along the pipeline without you clicking anything.
• The process daemon (daemon.py) — the long-running loop that keeps live state alive: streaming prices, enforcing hard risk controls, and reconciling exchange positions. The backend hosts it in-process by default (a separate START_DAEMON=1 process takes over if you run one). It runs whether or not autonomous research is enabled. The research loop rides on top of it.

The research loop never trades. It only researches: it proposes ideas, tests them, and changes a strategy's lifecycle stage. Execution is a separate concern handled by the scanner and the execution kernel, which this page contrasts below.

How it differs from the live scanner

This is the most common point of confusion, so it is worth stating plainly.

The daemon's job ends when a strategy graduates to paper. From there the scanner drives it through the execution kernel on every bar. Think of the daemon as the laboratory and the scanner as the trading floor.

The research loop, stage by stage

The daemon drives the real pipeline:

quick_screen → gauntlet → paper → live_graduated → archived

The public site simplifies these to screen → gauntlet → candidate → live, where candidate is the paper stage. The app uses the real names; you will see them in the strategy event log.

1. Ideation. When the active research pool runs dry, the planner emits a propose_crucible task. A crucible is a market thesis — indicators, regime, timeframe, assets, and a claimed edge. Crucibles can be invented by an agent (origin=agent), harvested from external sources, or seeded by you.
2. Refine and develop. The planner dispatches refine_crucible to detail the thesis, then develop_candidate to spawn child strategies under it, spending a shared daily research budget on the most promising crucibles. A data preflight runs first, and three fruitless attempts park a crucible so a bad idea cannot churn forever.
3. Quick screen. Every candidate hits the quick_screen gate first. Terminal failures (duplicate, overfit, or too few trades with an undefined Sharpe) auto-archive the strategy with no re-queue. Non-terminal holds — like a data-quality issue — are re-queued.
4. Gauntlet. Survivors enter the gauntlet, where the daemon auto-runs the readiness steps: timeframe sweep, optimization, apply-best-params, confirmation backtest, and the validation suite (walk-forward analysis, Monte-Carlo, parameter jitter, cost stress, and regime split). The output is a composite robustness score. See the gauntlet for the battery in detail.
5. Promotion to paper. When all gates pass, the daemon queues a promotion. By default this requires an operator approval; with auto_approve_promotions enabled it proceeds automatically. Paper is local-simulation only — no real capital.
6. Paper graduation. The paper-graduation job checks each paper strategy against the strict paper-to-live gate (days, closed trades, return, drawdown, forward Sharpe and profit factor). If it passes, a promotion into live_graduated is queued for your explicit GO LIVE confirmation — going live is never automatic.
7. Retirement. Strategies that decay — fitness below the floor or drawdown over the ceiling — are archived with a reason. Their linked quant skills take a confidence hit, closing the learning loop.

The thresholds behind each step are the stance preset you chose in Settings → Lab; treat any numbers as illustrative, not guarantees.

Relation to the brain and agents

The daemon does not think for itself. The brain is the sole orchestrator: it evaluates quality gates, decides lifecycle transitions, and dispatches work to specialized agents (strategy-developer, simulation-agent, risk-manager, quant-researcher). The daemon's scheduled jobs are the heartbeat that triggers the brain to act on schedule.

• The planner decides what research to do next and queues tasks.
• Agents carry out those tasks — writing strategy code, running backtests — and return results to the brain.
• The brain judges the results against the gates, records the decision and its outcome, and outcomes move the confidence of the quant skills that shaped the idea.

Agents never task each other and never place trades. Every capital-consuming transition the daemon proposes flows through the approval queue unless you have explicitly relaxed the policy.

Controlling the daemon

The daemon's aggressiveness is set by your autonomy mode, not by a single on/off switch. See autonomy modes for the full model:

• Manual (the default) — autonomous research jobs are frozen. The daemon keeps the process loop alive (prices, risk, reconciliation) but does not invent or promote on its own.
• Semi — the system stops minting new crucibles on its own, but fully researches, tests, and promotes the ones you enter.
• Auto — the full loop runs: ideation, testing, promotion, and learning.

Autonomous discovery has its own switches, independent of the mode:

// research settings (config.json / Settings → Lab)
{
  "autonomous_discovery": {
    "enabled": false,               // default: discovery is off
    "mode": "operator_approves",    // or "autonomous"
    "max_open_discovery_tasks": 1   // cap on concurrent harvests
  }
}

With enabled: false (the default) the daemon never harvests external ideas on its own — you trigger discovery manually with Harvest on the Crucibles page. With mode: "operator_approves", harvested crucibles land as proposed for your review before they advance.

Steps — turn on autonomous research

1. Connect an LLM provider on the Agents page — research needs one.
2. Pick a throughput preset that matches your provider's limits.
3. In the control strip at the top of the page, set the system mode to Semi or Auto. Manual freezes the research loop. See operations.
4. (Optional) In Settings → Lab → Research, enable autonomous_discovery if you want the daemon to harvest fresh ideas, not just test the ones already in the pool.
5. Leave the app running. The daemon works on its scheduled cadence; watch Pipeline → Processes and the Pipeline board.

What you'll see: new strategies appear in The Forge at the quick_screen and gauntlet stages, the activity feed logs planned and assigned task counts, promotions queue up in Approvals (unless auto-approved), and the Crucibles page fills with crucibles and their verdicts.

Manual triggers

You do not have to wait for the schedule:

# Operator-triggered crucible discovery (the Crucibles page's Harvest button)
curl.exe -X POST "http://127.0.0.1:8003/api/hypotheses/discover"

# Queue another research task for one crucible (Re-research)
curl.exe -X POST "http://127.0.0.1:8003/api/hypotheses/$HypothesisId/research"

# Run any scheduler job now
python -m forven scheduler run <job_id>

Discovery returns { created, task_id?, mode, reason? }. These routes are operator-gated; authentication and the full endpoint catalog are in the API reference.

The process daemon (daemon.py)

The research loop only runs because the process daemon keeps live state healthy underneath it. Its tick, in brief:

• Price feed — keeps a live price feed for the markets in play — Binance by default for paper, HyperLiquid for live — falling back to polling every ~15s, and publishes a shared price snapshot (before each risk cycle, so marks are never stale) that the scanner consumes. A mark watcher fills stops and take-profits at the touch moment.
• Risk cycle — recomputes real-capital equity against the high-water mark. If drawdown reaches max_drawdown_pct on three consecutive readings, the kill-switch trips, flattening all live positions autonomously (no approval) and re-sweeping every ~60s for residuals. Passing max_daily_loss_pct triggers the daily halt. Paper is never halted.
• Liquidation watch — every ~60s it measures each position's distance to its liquidation price and alerts when it crosses a warning (< 15%) or critical (< 7%) band.
• Reconciliation — roughly every 600s (sooner after a transient error) it compares exchange positions against the local books; on a mismatch it blocks new entries and alerts you. A single empty read is never treated as "everything closed".
• Freeze detection — if the daemon's own heartbeat goes stale past daemon_tick_max_stale_seconds (default 600), it goes red and alerts rather than failing silently.
• Heartbeat — every ~5m it logs open/closed trade counts, prices, sentiment, and regime to the activity table, and to Discord if configured.

These controls are hard and autonomous. The kill-switch acts without waiting for you. See risk & safety for how to read and respond to these states.

Caveats

• Paper is the supported mode, and promotion into live always needs your explicit confirmation; live is unsupported and at your own risk.
• The crucible planner and the hypothesis promotion loop both dispatch develop_candidate work from one shared daily budget; their de-duplication is best-effort, and under heavy load you may see redundant tasks in the queue.
• Autonomous discovery dedup against known crucibles is best-effort — an LLM under load can occasionally re-harvest a near-duplicate thesis.
• Daemon health is visible in health monitoring via the heartbeat; if the heartbeat goes stale, see troubleshooting before assuming the loop has stopped.

Forven is a research tool. The daemon's backtests, robustness scores, and shortlists are illustrative process output, not predictions of profit, and nothing here is financial advice. No strategy the daemon produces touches real capital without surviving the gauntlet and your explicit approval.

Related

• Crucible discovery — how new ideas are invented and harvested
• Hypothesis-driven research — the theses the daemon spawns and judges
• The brain — the orchestrator the daemon's jobs trigger
• Autonomy modes — how to freeze, harden, or fully unleash the loop