Build the complete system from scratch as a deployable Modal...

Prompt

Build the complete system from scratch as a deployable Modal application for one-GPU inference of a large joint video-and-stereo-audio diffusion-transformer model. Do not ask questions, omit no files, and do not provide a partial implementation. Produce every required source file and all required file contents. The result must run on Modal with exactly one GPU per container and must expose a usable HTTP API. Use a single NVIDIA GPU and select the best available price/performance/quality profile automatically from this priority: prefer B200 or B300 when the model fits resident in GPU memory; otherwise use the least expensive GPU that can execute the configured offload profile without changing output semantics. Permit configuration through environment variables, but default to Modal’s B200-or-newer-compatible allocation. The implementation must be CUDA 13-compatible and must support B300, whose runtime requires CUDA 13.1 or newer. Never require multiple GPUs, distributed process groups, NVLink, NVSwitch, tensor parallelism, sequence parallelism, Ulysses, Ring Attention, or inter-GPU communication. The target model is a roughly 33-billion-parameter multimodal diffusion-transformer that jointly denoises video and stereo-audio latents in one shared diffusion process. It accepts text, an optional reference image, an optional first frame, an optional last frame, and ordered multimodal reference inputs. It produces an MP4 containing video and synchronized stereo audio. Audio must be generated by the same denoising process, not dubbed after video generation. Support text-to-video-audio, image-to-video-audio, first-frame-to-video-audio, first-and-last-frame-to-video-audio, and reference-to-video-audio workflows. Also support image-only generation when the model pipeline natively supports it. Implement these fixed product constraints: - 24 frames per second. - Preset output tiers for 480p, 768p, and 2K, with normalized aspect-ratio choices. - Frame counts must obey the model’s valid frame-grid rule: frame_count = 17*n + 5 for a nonnegative integer n. - Reject unsupported dimensions, aspect ratios, frame counts, durations, reference counts, media types, and malformed requests with explicit HTTP 4xx responses. - Preserve synchronized video and stereo audio in the final MP4. - Support deterministic seeds. - Support positive prompts, optional negative prompts where supported, inference-step selection, guidance parameters where supported, and output format controls only when the underlying model supports them. - Support custom low-rank adapters. Load, validate, scale, merge, and cache adapter weights without changing the model’s public request semantics. Adapter merging must use W_new = W_base + scale × delta_W. Re-quantize merged weights into the active execution representation when necessary. Do not claim that this is free; implement it correctly and avoid corrupting resident base weights. Use the native model partitions and pipeline components rather than a remote model API. The model must be loaded from a configurable local or Hugging Face-compatible model identifier and cached in a persistent Modal Volume. Define the exact expected component layout and loading behavior for the text/vision conditioning encoder, diffusion-transformer, video VAE, audio VAE or audio decoder, vocoder if required, tokenizer/processor, scheduler, and media muxing path. Do not hard-code credentials or require authentication. Create exactly the necessary project files, including at minimum: - app.py: Modal image, Volume, GPU selection, container lifecycle, HTTP endpoint registration, warm-container model initialization, request parsing, inference dispatch, media upload handling, and response serialization. - modal.yaml: Modal application configuration, image/build settings, volume mount, function settings, HTTP exposure, timeout, and GPU configuration. - requirements.txt: pinned compatible Python packages for the selected CUDA/PyTorch/diffusion/runtime stack. - model_runtime.py: model loading, dtype selection, memory planning, component residency, CPU/layer offload, adapter handling, inference execution, and cleanup. - schemas.py: complete request and response schemas, validation, normalization, supported modes, resolution presets, frame-grid validation, and error types. - pipeline.py: text conditioning, image/frame/reference preprocessing, packed video/audio latent construction, shared denoising loop, scheduler integration, guidance, seed handling, latent decoding, and output assembly. - attention_backend.py: the fastest correct single-GPU attention implementation available for the selected hardware, with a backend selector and safe fallback. Where supported, implement FP8 attention using fused input preparation, packed/transposed value layouts, warp-specialized pipelining, numerically stable softmax, CUDA graph compatibility, and selective higher-precision residual repair. Do not invent undocumented kernel behavior; use only APIs and kernels that actually exist in the chosen dependencies. If a custom kernel cannot be implemented reliably from available libraries, use the fastest real fused attention backend and keep the interface identical. - memory_manager.py: static shape buckets, preallocated buffers, pinned host buffers, dtype-aware residency, layerwise/module offload, activation lifetime management, and peak-memory protection for one GPU. - media.py: GPU-side or hardware-assisted VAE decoding where supported, audio decoding/vocoding, frame conversion, stereo preservation, H.264/H.265 encoding selection, AAC audio encoding, MP4 muxing, temporary-file management, and byte-stream response creation. Avoid unnecessary device-to-host copies and pipeline decoding, encoding, and muxing where practical. - storage.py: model and adapter retrieval from the configured model source, persistent cache paths, atomic downloads, cache validation, and local input/output file handling. - errors.py: structured client and server error classes and conversion to JSON HTTP responses. - Dockerfile or Modal image definition: all system packages, CUDA-compatible runtime dependencies, FFmpeg with H.264/H.265 and AAC support, and reproducible Python setup required by the application. The HTTP API must provide one primary POST endpoint for generation. Accept JSON for text-only requests and multipart form data for requests containing images, frames, audio references, or adapter files. Define the complete request schema, including mode, prompt, negative_prompt, width, height, fps, duration or frame_count, seed, inference_steps, guidance_scale, resolution_tier, aspect_ratio, first_frame, last_frame, reference_inputs, adapter specifications, output format, and return behavior. Define a stable response containing the generated media as a downloadable file response or a local result URL produced by the application. Do not add user accounts, authentication, authorization, API keys, access-control rules, billing, quotas, rate limits, usage limits, product limits, audit logs, security policies, monitoring, or analytics. Implement a cold-start and warm-start strategy suitable for Modal. Load the model once per container, keep the largest reusable components resident, and use explicit locks so concurrent requests cannot corrupt shared model state. Since this is a one-GPU service, serialize GPU-bound inference unless a proven memory-safe scheduling strategy is implemented. Use request shape buckets so CUDA graphs and memory plans can be reused for identical normalized shapes. Do not batch requests with different shapes. Do not implement speculative or undocumented batching behavior. Use mixed precision deliberately: retain numerically sensitive conditioning, scheduler, normalization, final latent conversion, VAE, and audio operations in BF16 or FP32 as appropriate; use FP8 only for operators and model components validated for the target GPU and model quality. Include runtime capability detection for compute capability, CUDA version, BF16/FP8 support, available VRAM, and encoder availability. Fall back safely to BF16/FP16 kernels when FP8 is unavailable or would violate the selected quality mode. Implement the shared multimodal denoising process explicitly. Pack video and audio latent tokens according to the model’s native ordering, preserve modality metadata and positional information, run every requested timestep, and unpack the final latent streams without losing synchronization. Do not replace joint denoising with separate video and audio passes. Do not assume a distilled four-step model unless the configured checkpoint explicitly contains the corresponding distilled weights; expose the step count as a validated configuration and use the checkpoint’s native scheduler and timestep requirements. Implement optional caching only when it is mathematically valid for the loaded pipeline. Permitted optimizations include reusable tokenization, conditioning embeddings, static shape plans, CUDA graph capture, preallocated buffers, and exact reusable preprocessing. Do not reuse stale denoising features across unrelated requests or timesteps unless the selected model/runtime explicitly guarantees correctness. Never call ordinary autoregressive KV caching a feature of this diffusion system. Implement adapter support without mutating the shared base model in a way that leaks one request’s adapter into another. Maintain immutable base weights, create or retrieve an adapter-specific fused representation, associate it with a deterministic adapter key, and release unused representations according to memory availability. If an adapter cannot fit or cannot be fused safely on one GPU, return a clear client error rather than using a placeholder implementation. Optimize the one-GPU path for real performance and predictable memory use: - Prefer resident weights when they fit. - Otherwise implement layerwise or module-level CPU offload using pinned host memory and asynchronous transfers. - Keep video and audio VAE components resident when possible. - Avoid loading both the full conditioning encoder and full diffusion-transformer simultaneously when memory requires offload. - Preallocate buffers for each supported normalized shape. - Use nonblocking transfers only with correctly synchronized streams and events. - Use CUDA graphs only for static shapes and static control flow. - Use fused normalization, activation, rotary-position, projection, and attention operators where real dependencies provide them. - Keep intermediate tensors in GPU memory through decoding whenever practical. - Use hardware video encoding when available and fall back to a correctly configured FFmpeg encoder. - Ensure output streaming does not block GPU cleanup or corrupt temporary files. Define all configuration defaults in one configuration module and validate them at startup. Include model identifier, revision, cache volume path, precision policy, offload mode, attention backend, resolution presets, frame limits, adapter cache size, FFmpeg settings, timeout, and GPU selector. Do not place configuration in comments or undocumented magic constants. The generated project must be complete and runnable after installing the declared dependencies, attaching the declared Modal Volume, and providing the model identifier. Do not include tests, documentation, README files, examples, comments, docstrings, TODO markers, mock services, fake services, placeholders, dummy implementations, no-op methods, unimplemented methods, omitted functions, omitted files, or explanatory prose outside the requested source files and configuration files. Do not mention, reproduce, proxy, or depend on any branded third-party hosted implementation. Use neutral descriptive names for all modules, classes, functions, endpoints, variables, and identifiers; never use a proprietary provider’s name in the reimplementation. Return the entire implementation as a file-by-file project, with every file’s complete contents and no abbreviated sections. Send back the complete code files with all fixes applied. Fix all of the listed errors one by one, actually fixing them. Keep the original imports, because the files exist. Write out every single character, do not shorten anything. Fix every error. Do not write anything else, only the complete code files, and there must not be any comments in them. I never allow any simplified mock, placeholder, dummy, simulated, or fake crap, and write the complete files as unshortened production-ready code in a code block. They must be 100% error-free, and send them as downloadable files.