# Binance Futures USDT-M Screener — WebSocket-Only Real-tim...

Prompt

# Binance Futures USDT-M Screener — WebSocket-Only Real-time screener for **all eligible Binance USDT-M perpetual pairs**, built exactly to the fixed spec: a two-stage pipeline that discovers the universe from WebSocket array streams, graduates pairs on a live volume burst, and only then ranks them by price change. ``` ALL USDT-M PAIRS → VOLUME FILTER → QUALIFYING PAIRS → PRICE CHANGE → RANKING ``` **Global constraint honoured: no REST HTTP calls.** No `GET /fapi/v1/exchangeInfo`, no REST klines, no REST 24h tickers. Discovery, volume and price all arrive as WebSocket frames from combined market-data streams. This is enforced at runtime by `src/rest-guard.mjs`, which instruments `fetch` and the HTTP client surface and counts any attempt; the count is shown in the dashboard and in `/api/config`. A live run reports `restAttempts: 0`. --- ## Run it ```powershell node src/server.mjs # dashboard on http://127.0.0.1:8787 ``` or with the launcher: ```powershell .\run.ps1 # default geometry (18m warmup) .\run.ps1 -Port 8791 -Threshold 60 .\run.ps1 -Fast # shortened geometry: watch a cold start in ~3 minutes ``` Options (environment variables): | Variable | Default | Meaning | | --- | --- | --- | | `SCREENER_PORT` | `8787` | dashboard port (localhost only) | | `SCREENER_THRESHOLD_PCT` | `40` | graduation threshold; **minimum 40** | | `SCREENER_EARLIEST_FLOOR_USDT` | `5000` | dust floor for `E` | | `SCREENER_LATEST_FLOOR_USDT` | `1000` | dust floor for `L` | | `SCREENER_GRADUATION_HOLD_SEC` | `900` | how long a graduated pair stays ranked | | `SCREENER_RANK_CAP` | `30` | rows kept per side | | `SCREENER_VOLUME_SOURCE` | `aggTrade` | `aggTrade` (exact 1s) or `kline_1m` (minute-snapped) | | `SCREENER_PRICE_SOURCE` | `markPrice` | `markPrice` (falls back to `aggTrade`) or `aggTrade` | | `SCREENER_SYMBOL_MARK_STREAMS` | `1` | `0` drops per-symbol `markPrice@1s` (roughly halves bandwidth) | | `SCREENER_HOSTS` | `wss://fstream.binance.com,wss://fstream.binancefuture.com` | stream hosts, in failover order | | `SCREENER_RETENTION_BUCKETS` | `1080` | verification aid: scales the whole geometry proportionally | Tests (deterministic, offline, no network): ```powershell node test/acceptance.mjs # 105 checks, including every Acceptance Check in the spec ``` --- ## What the dashboard shows - **Status**: `LIVE`, `STALE — reconnecting` (evaluation frozen, partial ratios never served), `GAP` (a hole was observed) or `CONNECTING`. - **Stream time (T)**, with the local-clock skew it deliberately ignores for bucketing. - **WARMUP mm:ss** — no ranking at all until 1080 completed buckets exist. - **Two ranked sides**: right/green `chg_rank > 0` sorted most-positive first, left/red `chg_rank < 0` sorted most-negative first, top 30 each, `chg_rank == 0` hidden. - **Per row**: symbol, `chg_rank`, `chg_early` (labelled display-only), `L`, `E`, `ratio`, graduation countdown. - **Threshold slider** (min 40, applies forward only) and live source/host/connection chips. - **Rolling-window strip** with the gap rationale, pair-state census (`WARMUP` / `INVALID` / `FLOOR_FAIL` / `BELOW` / `QUALIFYING`), graduation log with `E, L, ratio, threshold, P_base, P_now`, operational log and per-connection table. --- ## Design decisions that matter ### One clock, anchored to the stream `T` is Binance **event time** floored to the second, never local wall time. This host's clock is skewed by months relative to Binance; bucketing trades by local time would mis-bin every trade, so `src/clock.mjs` owns the timeline. Local time is used only for diagnostics and the skew readout. The clock also holds back `LOCK_HOLD_SEC` (3s) before a second is lockable, so a frame that arrives late is still bucketed by *its own* event time rather than by arrival time. ### MISSING is a value, not a zero A silent second becomes `0` only when the feed provably carried data through it. Evidence is tracked two ways: - the clock keeps a set of **holes** (merged ranges) fed by frame-to-frame discontinuities and by socket drops, so a frozen or dead feed can never look like a quiet market; - each shard records the event-time interval it actually delivered, and every per-symbol ring asks *its own* socket whether that second was covered. Anything else is `MISSING`, and any window containing `MISSING`, an open bucket or a late-touched bucket is invalid — the pair is skipped rather than evaluated on partial data. Nothing is ever interpolated, carried forward, or backfilled (there is no backfill: recovery is live-only and can take up to 18m, or more if the hole itself has to age out of the span). ### Why the 2-minute gap exists `E = [T-18m, T-3m)`, then a retained-but-unused 2m band, then `L = [T-1m, T)`. The middle band is kept so the price endpoints `P_early` (T-18m) and `P_base` (T-3m) and window continuity keep working, but it is excluded from the volume ratio so one decaying spike sitting next to `L` cannot both inflate the latest window and deflate the baseline. Uniform flow therefore reads `1/15 ≈ 6.67%`, and the 40% default is a ~6x pace burst. ### Edge-triggered graduation `graduated_until = now + 15m` is set only on a **crossing** from not-qualifying to qualifying. While the ratio stays above threshold the timer is never touched, so there is no rolling extension; re-arming requires the ratio to fall below threshold (or become invalid) and cross back above. A graduated pair keeps its existing expiry even if the threshold is raised. ### Prices come from one source per evaluation Default source is `!markPrice@arr@1s`; the fallback is the last trade price from the same `aggTrade` stream used for volume (no extra subscription). Both are tracked per symbol and the UI labels which one produced the row. Sources are never mixed inside a single evaluation, and an endpoint older than `PRICE_STALE_SEC` (30s) skips the pair's ranking rather than reusing a stale price. ### Stream transport `!miniTicker@arr` drives discovery, and per-symbol `aggTrade` (+ `markPrice@1s`) streams are subscribed dynamically as the universe changes. Binance allows up to 1024 streams per combined connection, so per-symbol streams are packed densely — a ~550-pair universe costs **7 sockets** rather than hundreds, which also avoids connection-rate limits. Batches are rate limited, empty param lists are never sent (Binance closes with 1008), and every socket has a watchdog that rotates to the next host if it connects but never delivers a frame. **Bandwidth.** Carrying every pair's `aggTrade` *and* `markPrice@1s` measured **≈1.8 MB/s (~6.5 GB/day)** for a ~550-pair universe, and the per-symbol mark streams are most of it. That is the price of an independent 1s price series per pair: set `SCREENER_SYMBOL_MARK_STREAMS=0` to drop them, and prices then resolve from `!markPrice@arr@1s` plus the `aggTrade` fallback — exactly the WS-only set the spec permits, at roughly half the bytes. Note that high per-pair rates never reach the browser: only graduated pairs (top 30 per side) are sent, once per second over SSE from the local process. --- ## Environment note: `fstream.binance.com` is blocked on this network This is worth knowing because it looks like a code bug and is not one. From this machine: | Endpoint | Result | | --- | --- | | `wss://fstream.binance.com/...` | TLS handshake and WS upgrade succeed, then **0 frames in 30s** | | `wss://fstream.binancefuture.com/...` | streams normally (all three families verified) | | `wss://stream.binance.com:9443/...` (spot) | streams normally | So the canonical futures host completes the handshake and then silently withholds data. The screener therefore treats the two official USDT-M futures hosts as failover peers, probes each socket for frames, and rotates hosts when a socket stays silent — the log shows the rotation, for example: ``` [HOST-FAILOVER] control#1 wss://fstream.binance.com delivered 0 frames in 10s -> wss://fstream.binancefuture.com ``` Set `SCREENER_HOSTS` to pin a single host if your network reaches the canonical one. --- ## Layout ``` run.ps1 launcher (-Port, -Threshold, -Fast, -Hosts) src/ config.mjs frozen spec constants + env overrides + window rationale text clock.mjs stream timeline: hold-back, seam lag, holes, proven coverage, STALE universe.mjs !miniTicker@arr discovery, N-push persistence, 30s eviction volume-ring.mjs per-pair 1s quote-volume ring: dedupe, late trades, MISSING, WARMUP price-store.mjs per-pair 1s price store: markPrice + aggTrade, backward-fill endpoints evaluator.mjs windows, floors, edge-triggered graduation, two-sided ranking stream-pool.mjs combined-stream manager: shards, SUBSCRIBE/UNSUBSCRIBE, host failover screener.mjs orchestration + UI snapshots rest-guard.mjs runtime instrumentation proving no REST is used server.mjs local dashboard server (127.0.0.1) + 1s evaluation loop public/ dashboard (vanilla JS, SSE from the local process only) test/ harness.mjs synthetic WS driver (mono clock, no I/O) acceptance.mjs 105 checks: window arithmetic, all spec acceptance checks, sorting tools/ screenshot.mjs CDP-driven dashboard capture (waits for live content) docs/ dashboard screenshots referenced above ``` ## Measured behaviour (live runs against Binance) **Cold start at the spec's own geometry** (1080s retention, every pair subscribed): ``` T+0:00 universe=551 conns=7 warmed=0/551 stateCounts WARMUP:551 T+18:00 warmed=551/551 stateCounts WARMUP:0 QUALIFYING:23 BELOW:143 FLOOR_FAIL:385 INVALID:0 graduations=56 ranked=49 (30 up / 12 down / 3 flat hidden) restCalls=0 worstCoverage=SLERFUSDT 1080/1080 T+21:00 1,586,538 frames / 2,276 MB in ≈ 1.8 MB/s clockState=LIVE the whole time, INVALID:0 graduations logged=78, currently ranked=61 (30 up / 22 down / 3 flat) host failover once, socket watchdogs 1, reconnects 2, restCalls=0 ``` Nothing graduated and nothing ranked for the first 18 minutes, exactly as required; the moment the last pair's 1080th bucket completed, pairs moved to `QUALIFYING` / `BELOW` / `FLOOR_FAIL` and real graduations began. `INVALID:0` after 21 minutes means not one window was ever evaluated on a hole, and `removals=0` means the 30s eviction rule never had to fire. A sample graduation line as logged by the engine: ``` [GRADUATE] BIGTIMEUSDT T=1791024026 E=1934217 L=1234598 ratio=63.83% threshold=40% P_base=0.00906 P_now=0.00908 (markPrice) until=1791024926 ``` **Shortened-geometry run** (`SCREENER_RETENTION_BUCKETS=180`, same code, same properties) was used to watch the whole path end-to-end in minutes: 221 graduations and two fully populated 30/30 ranked sides, `restAttempts: 0` throughout. `restAttempts: 0` in every run — the guard instruments `fetch` and the HTTP client surface, so "no REST" is a measured property here, not an assertion. The dashboard after warmup is captured in `docs/ui-ranking.png`; the cold-start state is in `docs/ui-live.png`.