ADR 0003 — Device memory budget and primary-visibility buffe...

Prompt

ADR 0003 — Device memory budget and primary-visibility buffers Status: accepted (budget: user decision; buffer formats: delegated; granularity: runtime parameter). Amended: 2026-09-23 (Amendment 1: image-identity rule); 2026-09-24 (Amendment 2: granularity default). Context and Constraints - Cap (P-001): Max 3.5 GB VRAM on RTX 3050 Laptop (4 GB physical). Driver heap budget (VK_EXT_memory_budget) measured at 3,367.7 MiB (~3.53 × 10⁹ B). - Requirements: Fixed device budget prior to allocation; G-buffer formats before raster (2C); shared ID scheme between raster and ray query for exact parity (2D). Decision 1. Device Budget (memory::device_budget): - \text{Total} = \min(3.5 \times 10^9\text{ B}, \text{driver heap budget}) - 10\%\text{ headroom} = 3.15 \times 10^9\text{ B} (~3,004 MiB). - Device/staging allocations require a Ledger grant. Refusals defer publication, keep the previous snapshot visible, and log the event (no silent evictions or overcommit). - Categories are reported globally; per-category quotas await measured contention. 2. Primary-Visibility G-Buffer (Raster 2C / Ray Query 2D parity): | Target | Format | Bytes/px | Content | | :--------- | :------------- | :------- | :------------------------------------------------------------------------------------------------------------------------------ | | Depth | `D32_SFLOAT` | 4 | Reverse-Z depth | | Normal | `R16G16_SNORM` | 4 | Octahedral unit normal (exact for axis-aligned faces; ready for smooth voxels) | | Material | `R16_UINT` | 2 | Exact `MaterialId` (unfiltered, unblended) | | Surface ID | `R32G32_UINT` | 8 | `(instance_index, primitive_index)` matching ray query `(InstanceCustomIndex, PrimitiveID)`; resolves to `(chunk, brick, quad)` | - Footprint: 18 bytes/px (37.3 MB at 1080p; 66.4 MB at 1440p). Snapshot version is a per-frame push constant. Motion vectors and history buffers defer to Phase 3. 3. Granularity Parameter: - Mesh merge: none vs. greedy within a brick. - Acceleration region: 1 brick, 2^3 bricks, 1 chunk (4^3 bricks), or 2^3 chunks. - Runtime configurable via benchmark sweep. Correctness requires cross-setting agreement at LoD 0, not golden-image matching. Contracts and Validation - Host/shader layouts verified against slangc reflection. - Implemented & tested: memory::device_budget verified against varying driver budgets, overcommit refusals, and laptop constraints. Amendment 1 (2026-09-23): LoD 0 Equivalence Rule Replaced "bit-identical image" with an explicit geometric agreement contract. Merged quads alter primitive IDs, depth varies by floating-point ulp across triangulations, and greedy T-junctions introduce single-pixel raster/ray differences. - Contract: For identical camera, snapshot, and LoD 0, all granularity settings and the ray-query path must match per pixel on: - Material ID (exact), normal (exact), resolved (voxel, face) pair via snapshot tables (exact), depth (within declared tolerance), crack pixels (zero). - Boundary pixels within depth/position tolerance of projected voxel edges are excluded from identity gates; excluded pixel counts are tracked and reported. Amendment 2 (2026-09-24): Granularity Default - Default: Greedy merge within brick, 1 acceleration region per chunk (--merge greedy --region chunk). Documented alternative: 2^3 chunks (--region 2x2x2_chunks). - Rationale: 2E sweep showed a trace-time plateau (0.91–1.0 ms at 1080p across all 8 settings). Greedy merge reduced memory to ~25% and cut raster cost by 40%. Chunk regions build in 3 ms and easily satisfy the edit budget (p95 \le 50\text{ ms} for 1^3 and 8^3 edits; \le 100\text{ ms} for 32^3). Change (Planned): Phase 3 — Honest Sampled Lighting and Reconstruction Status: proposed, not authorized. (Builds on Phase 2, M1/M2 accepted). 1. Current State - Materials (world::material::MaterialParams) contain only linear base colour and unstandardized emissive colour; no physical units, roughness, glass, or water semantics. - Lit view (View::Lit) is a debug ambient/diffuse shade. - Reusable Assets: G-buffer (18 B/px), raster/ray-matched TLAS, snapshot edit swap, CPU voxel DDA geometry oracle (world::reference::trace). - Missing Assets: Motion vectors, history ping-pong buffers, reset state, physical light units, light lists, reference path tracer. 2. Scope and Principles - Implement light transport, BSDFs, emitter sampling, MIS, reference path tracer, temporal accumulation, guide schema, and native reconstruction. - Gate: Transport and temporal guide/reset behaviours must be provably correct before adding reuse structures (Phase 4). The reference and real-time paths are evaluated independently. - Execution order: emissives/sun \to sky \to temporal \to reconstruction \to indirect bounce. 3. Proposed Slices - 3A — Conventions and Reference Renderer: - Define ADR-0005: physical units (sun normal irradiance, sky radiance, emission), Lambertian BSDF (albedo \le 1), fixed secondary ray origin offsets, solid-angle PDFs, balance-heuristic MIS, documented RNG seeding. - GPU reference: progressive path tracer (f32 accumulation) via TLAS/materials. Cross-checked on small viewports against CPU DDA reference. - Tests: white furnace convergence to L_e / (1 - a); analytical sun-plane match; GPU vs. CPU reference convergence; negative tests for planted PDF/cosine errors. - 3B — Sun and Shadows: - 1 shadow ray/px from G-buffer surface. - Hard-shadow control validated against CPU DDA shadow rays (with edge exclusion). - Real-time disk sun sampled at 1 spp; accumulated frames must converge to reference direct-sun component within 3\sigma. - 3C — Sky: - Analytic sky model with cosine-weighted hemisphere rays (1 spp), MIS-weighted against the sun disk. No heuristic AO; occlusion is fully visibility-driven. - 3D — Temporal Foundation: - ADR-0006 guide schema: depth, normal, material, surface ID, motion vectors, albedo (for demodulation). Previous frame G-buffer kept via ping-pong. - History tracking: age, validity, and explicit rejection flags (disocclusion, normal/depth divergence, material/surface ID change, region edit version, camera cut). - Tests: static convergence; zero-age persistence under planted resets; ghosting detection under planted bypass; edit resets strictly localized to edited regions. - 3E — Native Reconstruction: - Demodulated spatio-temporal filter (SVGF-style) guided by G-buffer channels, followed by re-modulation. - Error budget measured against progressive reference: mean luminance bias within budget (no filter energy loss); edge sharpness preserved across material boundaries. - 3F — One Bounce Indirect Diffuse: - 1 continuation ray/px evaluating sun and sky at secondary hit. Accumulated result must match 1-bounce reference limit. Higher bounces tracked as known residual error. - 3G — M3 Gate: - Street loop performance benchmark (p99 \le 16.7\text{ ms} at 1080p). Reference error bounds verified on fixed paths. Edit latency verified with localized history resets. (Deferred past M3: emissive polygon light lists, glass/water semantics, ReSTIR reservoir reuse, DLSS/RR). 4. Budgets - Device Memory (1080p target): | Target | Format | Bytes/px | | :------------------------------- | :------------------- | :----------------------- | | Previous G-Buffer | Mixed (Table 2C) | 18 | | Motion | `RG16F` | 4 | | Noisy Illumination | `RGBA16F` | 8 | | History Illumination (Ping-pong) | `RGBA16F` $\times 2$ | 16 | | Moments & Age (Ping-pong) | Mixed | 12 | | Filter Scratch (Ping-pong) | `RGBA16F` $\times 2$ | 16 | | **Real-time Total** | | **\~74 B/px (\~155 MB)** | | Reference Accumulation | `RGBA32F` | 16 B/px (\~33 MB) | - Fits within the 3.15 \times 10^9\text{ B} device budget. - Ray Budget: 4 traced rays/pixel in real-time path (1 primary raster + 1 sun + 1 sky + 1 bounce continuation + 2 shadow/visibility at bounce hit). 5. Decisions Needed from User 1. Lighting Order: Sun/sky first (Slice 3B/3C), leaving emissive street lamps for post-M3. Provides clean analytical baselines with minimal noise. 2. M3 Gate: Lit street under sun/sky with 1 bounce, validated against reference, maintaining M1 perf (p99 \le 16.7\text{ ms} at 1080p MAILBOX). 3. Error Metric: Relative MSE and mean-luminance bias (\le \pm 2\% vs. reference) via existing tooling. 4. Reconstructor: Native SVGF-style spatio-temporal filter on demodulated lighting. 5. Sky Model: Parameterized analytic sky model behind a swappable interface. 6. Authorization: Authorize 3A (conventions ADR and reference renderer). 6. Proposed Work Units | Slice | Scope | Units | | :-------- | :------------------------------- | :----- | | **3A** | Conventions & Reference Renderer | 6 | | **3B** | Sun & Shadows | 4 | | **3C** | Sky Visibility | 3 | | **3D** | Temporal Foundation | 6 | | **3E** | Native Reconstruction | 7 | | **3F** | Single Bounce Indirect | 4 | | **3G** | M3 Integration & Gate | 3 | | **Total** | | **33** | 7. Key Risks - Look Divergence: Moving to radiometric units alters exposure and tone; debug view retained as baseline. - Filter Softening: Demodulated SVGF filtering must satisfy luminance and edge budgets to prevent energy loss or excessive blurring. - Secondary Ray Divergence: Incoherent secondary rays on laptop hardware require early cost validation in 3B. - Edit Invalidation: Region-level history invalidation can cause visible lighting pops; invalidations must be culled strictly to modified brick bounds rather than resetting entire acceleration regions. The proposal in short. Proposed milestone M3: "the street lit by sun and sky, with real shadows and one bounce of light, in real time, checked against a reference." It's split into 33 work units: 3A (6): a doc fixing light units and the shading model, plus a slow, accurate reference renderer that everything else is checked against. 3B (4): sun shadows. A sharp-shadow mode must match the CPU ray tracer exactly, pixel for pixel, the same kind of check as raster versus ray query. The normal mode uses a soft-edged sun. 3C (3): sky light. Rays toward the sky also give the darkening in corners and under awnings, so there's no separate shortcut for that. 3D (6): tracking what carries over between frames: motion, how old each pixel's history is, and why it was thrown away. Tests confirm this can catch "resets every frame" and ghosting before any denoiser is tuned. 3E (7): our own denoiser, checked against the reference so it can't hide lost light or blur away detail. 3F (4): one bounce of light, so shadowed areas aren't wrongly dark. 3G (3): the M3 gate: speed, quality against the reference, edits, and you walking the street. Street lamps and the shop sign at night, then glass, then water, come after M3. The new per-screen buffers are estimated at about 155 MB at 1080p, well inside the memory budget. No new dependencies are needed. Decisions for you (proposal §5): Day before night? Your Phase 0 answers put glowing materials (lamps, sign) first. I recommend sun and sky first, because sharp sun shadows can be checked exactly and they build and test the shadow, reference and history code cheaply. Many small lamps are noisy at one sample per pixel and need that code anyway. What counts as M3 being met: I suggest the same 60 fps check as M1 (1080p, 99% of frames under 16.7 ms) plus the quality check against the reference. If it's too slow, you decide whether to lower resolution or sample count; I won't do it quietly. How to measure denoiser quality: recommended is average error plus overall brightness within ±2% of the reference, with no new tools. NVIDIA's FLIP image comparison is optional and needs your approval. Denoiser: our own, in the SVGF style. DLSS and Ray Reconstruction stay last, as you asked in Phase 0. Sky: a simple formula first, replaceable by a physically based sky model later. Go-ahead for 3A: the light-conventions doc and the reference renderer.