Skip to content

Add provider-neutral historical-to-realtime streaming#85

Merged
luisleo526 merged 5 commits into
mainfrom
codex/alert-webhooks
Jul 10, 2026
Merged

Add provider-neutral historical-to-realtime streaming#85
luisleo526 merged 5 commits into
mainfrom
codex/alert-webhooks

Conversation

@luisleo526

@luisleo526 luisleo526 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a continuous confirmed-OHLCV warmup → realtime normalized-trade lifecycle
  • preserve positions, equity, pending orders, Pine/TA state, request.security() state, and partial timeframe aggregation across the handoff
  • evaluate resting broker orders on every normalized trade while retaining the default close-only strategy calculation cadence
  • expose a provider-neutral four-field trade POD through the stable C ABI:
    • source event timestamp
    • optional monotonic normalized sequence
    • executed price
    • quantity in the configured symbol's volume units
  • keep provider authentication, transport, symbol mapping, and provider-only metadata outside the engine
  • skip quiet intervals outside the configured syminfo session instead of manufacturing tradable bars across market closures
  • expose the lifecycle through the Python ctypes harness and contiguous mmap replay tooling
  • add a dedicated cdocs streaming guide and runnable tutorial

Why

TradingView strategies load confirmed historical bars and then continue on realtime data without replacing the strategy instance. PineForge previously only exposed one-shot backtests, so switching data sources could not preserve the complete broker and script state.

This PR introduces the engine-owned lifecycle needed by a separate data layer such as pineforge-data. Any community provider adapter can normalize its payload into pf_trade_tick_t; the engine remains deterministic, dependency-free, and unaware of HTTP, WebSockets, credentials, or vendor schemas.

It also establishes the realtime execution foundation needed for later alert(), order-fill alert, and JSON webhook work. Alert delivery itself is intentionally not part of this PR.

Validation

Current head (1226c64):

  • ctest --test-dir build --output-on-failure: 85/85 passed, including provider-neutral ABI coverage, session-aware gap handling, and test_tutorial_stream
  • ./scripts/run_corpus.sh: 252/252 executed; 239 excellent, 12 strong, 1 existing anomaly; no moderate/weak/minimal results
  • python3 scripts/check_c_abi_runtime.py: passed
  • bash docs/build.sh: generated without warnings
  • Python syntax checks for the ctypes harness, both stream corpus runners, and tutorial: passed
  • git diff --check: passed

Existing 24×7 raw-trade replay evidence for the streaming lifecycle:

  • 756/756 sessions completed
  • 756/756 input/script bar-count parity
  • 750/756 trade-count parity
  • 729/756 exact ordered trade structure
  • 99.866844% weighted ordered trade-structure parity across 4,463,189 compared trades

The long replay used three session starts (2025-03-29, 2025-03-30, and 2025-03-31), each ending exclusively at 2025-05-01. Each strategy/session received one contiguous strategy_stream_push_ticks(pointer, count) call.

@luisleo526 luisleo526 changed the title Add continuous historical-to-realtime strategy streaming Add provider-neutral historical-to-realtime streaming Jul 10, 2026
@luisleo526 luisleo526 marked this pull request as ready for review July 10, 2026 08:41
@luisleo526 luisleo526 merged commit 9734d48 into main Jul 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant