Build from scratch a complete, self-contained, mixed-modalit...
Prompt
Build from scratch a complete, self-contained, mixed-modality quantization, compression, fine-tuning, inference, and vector-retrieval system named precision_engine in Python 3.11+, PyTorch, C++17, and CUDA C++. Implement every file and kernel completely. Exclude everywhere: authentication, authorization, access control, permissions, security policies, auditing, monitoring, compliance telemetry, rate/usage/product limits, quotas, billing, payments, tests, benchmark harnesses, documentation, READMEs, docstrings, comments, placeholders, mocks, dummy code, fake services, stubs, omitted files or functions. Never mention original third-party software, method, vendor, or model brand names; use only the neutral identifiers below. 1. FILES AND BUILD CONFIG Create all 36 files: Root: pyproject.toml, setup.py precision_engine/: __init__.py, cli.py precision_engine/core/: __init__.py, formats.py, transforms.py, trellis_lattice.py precision_engine/calibration/: __init__.py, data_builder.py, scoring.py precision_engine/solvers/: __init__.py, hessian_solvers.py, discrete_solvers.py, annealing_and_transfer.py, specialized_modalities.py precision_engine/training/: __init__.py, qat_and_distill.py, adapters_and_optim.py precision_engine/search/: __init__.py, recipe_controller.py precision_engine/runtime/: __init__.py, kv_and_state.py, moe_and_layers.py, decoding_controller.py, engine.py precision_engine/vector_index/: __init__.py, tiered_index.py precision_engine/io/: __init__.py, container.py csrc/: bindings.cpp, hadamard_kernels.cu, trellis_lut_gemm.cu, mixed_tile_and_moe_gemm.cu, kv_and_state_kernels.cu, vector_scan.cpp pyproject.toml: PEP 517 setuptools/wheel config, deps torch>=2.3.0, numpy>=1.26.0, scipy>=1.11.0, script precision-engine = precision_engine.cli:main. setup.py: CUDAExtension precision_engine._C from all 6 csrc/ files with C++17 -O3, NVCC -O3 --use_fast_math, sm_80/sm_89/sm_90. Each __init__.py exports package symbols. Zero comments. 2. FORMATS AND GRIDS (precision_engine/core/formats.py) Bit-exact pack, unpack, quantize, dequantize: - Primitives: symmetric/asymmetric INT1-INT8, ternary {-1,0,+1}, E2M1 with subnormals, E4M3, E5M2, block scales UE8M0 (power-of-two), UE5M3, signed E4M3. - CompandingOffsetGrid: closed-form 2^b-level grid with companding strength alpha in [0,1] (uniform spacing at 0 to float tail spacing at 1) and asymmetry offset c in [-1,1]. Reuse unused sign bit of E4M3 scale to select G(alpha,c) vs mirror -G(alpha,c), doubling distinct values per block at zero bit cost. fit_companding_offset_grid fits (alpha,c) for 2-6 bits via grid search plus Nelder-Mead. - LogBaseQuantizer: q(x) = sign(x) s b^clamp(round(log_b(max(|x|/s,eps))), -2^(b-1), 2^(b-1)-1), learning base b>1 per tensor/channel for heavy-tailed weights. - ZeroCal4BitFormat: calibration-free layerwise 4-bit quantization over size-64 row sub-blocks via quantile/norm clipping search, packing 4-bit indices with double-quantized scales. - CausalBoundedQuantizer: deterministic causal quantizer keeping synchronized adaptive tables C_t on encoder/decoder (init >=eps spacing or uniform 2 eps step), guaranteeing |x_t - x_hat_t| <= eps from x_<=t alone and updating C_(t+1) strictly from transmitted symbols. 3. TRANSFORMS (precision_engine/core/transforms.py) PyTorch ops and precision_engine._C dispatchers: - fwht, block_hadamard: normalized O(d log d) Fast Walsh-Hadamard Transform for blocks 32, 64, 128, 256, 1024; relocate_subspace_hadamard runs inverse H_128 on activations so weights stay exact integers. - DitheredRHT: sparse random diagonal signs D, Hadamard H, calibrated non-uniform pre-rounding dither, matching subtraction on reconstruction. - OutlierSignAdjustedRotation: single pass finds fixed massive outlier channels (especially MLP output projection inputs post-nonlinearity), builds sign vector s in {-1,+1}^d unifying target Hadamard row signs to collapse bimodal distributions into unimodal profiles, pairs with asymmetric activation quantization absorbing shifted mean via zero-point, applying only element-wise sign flips before fwht at inference. - CayleyOrthogonalRotation: skew-symmetric A = P - P^T, orthogonal Q = (I-A)(I+A)^-1 plus positive channel scales on query, key, value, gate, up projections, optimized on activations. - PCASubspaceRotation: activation SVD basis V, rotate activations, keep top-k coordinates in FP16/BF16, quantize rest to 4 bits, fuse inverse V offline into weights. - LayerLocalTransform: per-channel balance scales and block-32 Hadamard inside each linear layer, absorbing inverses into weights with input hooks and zero cross-layer propagation. 4. TRELLIS AND LATTICE CODECS (precision_engine/core/trellis_lattice.py) - ComputedTrellisCodec: single shift-register decoder for 1-6 bits/element. Multiplies state by two integer coefficients, sums top bits of both products into non-uniform 6-bit index over a 64-byte table whose rare ends store tail magnitudes and dense center covers core; includes INT/FP presets; encode_trellis_viterbi minimizes Hessian error. - SubspaceIntTrellisCodec: Viterbi matching on 128x2 tiles (256 weights) aligned with H_128 constraining outputs to exact INT4/INT8 codewords for integer GEMM. - PolarE8VectorQuantizer: precondition 8D blocks via random signs and normalized Hadamard, split into magnitude (1D Lloyd-Max) and unit direction, build codebook from normalized E8 shells on S^7 (augmenting from max-error blocks if <2^b) indexed by leader, permutation rank, signs; proxy-Hessian H_cc ||W_[:,c]||_2^2 keeps top 1% sensitive channels at high precision. - Lattice24Quantizer: encode 24D blocks via binary Golay coset nearest-point decoding on 24D Leech lattice (~2.7 bpp overall without large codebooks). - MicroScaledTrellisQuantizer: beam-search trellis encoder plus scale-conditioned hash from block-normalized Gaussian CDF outputting exact E2M1 or UE5M3 elements. 5. CALIBRATION AND SCORING (precision_engine/calibration/) - data_builder.py: - build_fragility_weighted_mixture: 2-bit retention scan across task families, rank fragile tasks by low retention, blend fragile samples with generic corpus tokens. - build_reasoning_opening_mixture: generate initial 256 tokens from full-precision model on reasoning prompts (capturing ~20% higher activation variance across initial ~30 steps), mix 9:1 with generic text, pad/truncate to 512 tokens. - scoring.py: - compute_hidden_kl_divergence: mean and 99th-percentile forward KL and top-1 agreement vs full-precision logits on held-out sequences. - detect_repetition_loops: sample long continuations and flag rolling n-gram loops. - compute_gradient_sensitivity: teacher-forced forward/backward passes over tail tokens computing (dL/dy)^2 (y_hat - y)^2 per module. 6. SOLVERS (precision_engine/solvers/) - hessian_solvers.py: - MismatchCorrectedSuccessiveSolver: H = 2 X X^T + lambda I and mismatch X_fp - X_q, orthogonal projection onto weight subspace for closed-form layer-adaptive interpolation, Cholesky inverse, greedy successive rounding or K-beam search on triangular least-squares. - JointScaleIntegerSolver: row warm-start across scale-refitted candidates, +/-1 integer flips recomputing closed-form optimal group scale coupled across row groups via full H, periodic joint row scale solve, diagonal Tikhonov regularization. - FlipRefinementStreamingSolver: lazy-load one linear layer at a time, optimize Hessian-weighted shared block scales, greedy coordinate descent flipping codes to adjacent E2M1 levels minimizing dw^T H dw. - discrete_solvers.py: - GumbelSoftmaxGridSolver: annealed Gumbel-Softmax relaxation over local-shift indices (3-8 levels) jointly optimizing logits and symmetric group scales without zero-points, hardening and projecting back into packed binary formats. - AdaptiveClusterLUTSolver: allocate K in {8,16,32} centroids per matrix via first-order importance under LUT latency budget, K-means init, freeze centroids, train low-rank soft assignment updates annealed to hard indices. - OutputKroneckerFisherSolver: rank-one Kronecker output Fisher approximation jointly optimizing ranges, transforms, clipping, and rounding. - annealing_and_transfer.py: - SimulatedAnnealingDecompositionSolver: calibration-free W approx Q + U V^T + S (rank-32 SVD plus 1% row-sparse S) over 300 geometric cooling steps. - WeightSpaceQuantVectorTransfer: v_q = theta_donor_qat - theta_donor_base, patch theta_target + lambda v_q before PTQ. - specialized_modalities.py: - Tile2DMixedPrecisionAllocator: hardware-aligned 2D tiles (16x16 or 32x32), score via activations or gradient sensitivity, keep top 1%-20% tiles at 16-bit and rest at 8-bit. - MoEPruneAndQuantizeAllocator: joint expert pruning and bit allocation under tight budgets keeping routers, shared experts, and attention in BF16, plus equivalent RMSNorm scale tuning preventing routing drift. - MonocularDepthQuantizer: joint Q/K attention-preserving calibration matching full-precision softmax maps plus channel affine alignment folded into weights. - WorldModelRolloutQuantizer: keep early encoder blocks and control channels in high precision (padded GEMM plus control buffer), calibrating encoder and per-step predictor scales jointly over rollouts. - ImageCompressionJointQuantizer: joint search of channel-balancing exponents and sequential Hessian weight updates under rate-distortion loss. - VariableRateSpeechCodecModulator: segment speech frames via latent variation density and modulate active codebook depth per segment. 7. TRAINING, ADAPTERS, AND OPTIMIZER (precision_engine/training/) - qat_and_distill.py: - QuantizationAwareDistillationTrainer: frozen teacher training quantized student via top-500 forward KL and deployment-informed one-hot cross-entropy on teacher traces. - ProgressiveMultimodalQATTrainer: teacher response self-relabeling, per-layer vision-sensitivity tau, progressive fake-quantization starting on vision-sensitive layers before early text layers. - adapters_and_optim.py: - CompactCoefficientAdapter, CompactAdapterLayer: residual adapter storing ~2.9M coefficients (~11 MB at 27B) modulating shared projections; kept unmerged in GPU memory with no-adapter slot, per-token routing buffer, and chunked workspace for single-kernel mixed-adapter batching. - ResidualSignSubspaceOptimizer: torch.optim.Optimizer subclass storing one BF16 momentum per 2D weight, updating top-r r-by-r singular subspace via AdamW moments and normalized sign updates from orthogonal residual outside subspace. 8. RECIPE SEARCH (precision_engine/search/recipe_controller.py) - ClosedLoopRecipeOptimizer: iterative search over transforms, solvers, grids, bit widths, group sizes, scale precisions (format-aware scale search and distillation), preservation masks, and calibration mixtures, scored by hidden KL and loop detection. - BitTieredAutoCalibrator: sweep cost = w_bits bits + w_rerank rerank_depth with w_bits > w_rerank (rerank_max - rerank_min) evaluating preconditioning off/on inside each tier before escalating bits. 9. RUNTIME AND DECODING (precision_engine/runtime/) - kv_and_state.py: - CenteredMicroScaledKVCache: subtract per-layer/head mean key from initial 256 prefill tokens before rotation/quantization; execute (a) rotated E8 split-KV flash decoding with rotated-domain V accumulation and single output un-rotation, (b) microscaled trellis KV decode, (c) E2M1+UE8M0 fused multi-query speculative verify across K draft queries and GQA groups with LSE merge. - WindowedRecurrentStateManager: quantized state S plus W-token high-precision update buffer for linear-recurrence layers, offload outlier rows/cols into high-precision compensator tokens sharing update paths, smooth residuals, value-axis grouping plus decay-aware mixed precision, stochastic rounding at window steps. - moe_and_layers.py: - NestedPrecisionMoELayer: pack experts in nested floor-and-compensation bit planes, dynamically choosing per-expert precision from router gating scores at runtime. - decoding_controller.py: - GlobalDivergenceSamplingCalibrator: label-free global (T_q, p_q) fit per requested temperature minimizing forward KL, reverse KL, or JSD vs full precision (zero inference overhead). - PerStepAdaptiveSamplingHead: two-layer MLP mapping hidden states and entropy at step t to token-level (T_t, p_t). - engine.py: autoregressive executor integrating transformers, MoE, linear-recurrence hybrids, mixed-adapter routing, speculative verify, adaptive decoding. 10. VECTOR INDEX (precision_engine/vector_index/tiered_index.py) - Subtract centroid mu, learn PCA basis U ordered by descending variance sigma_j^2, greedily allocate b_j in {0..12} bits per coordinate maximizing MSE reduction per bit, encode optional fine-tier residuals, pack variable-bit codes with alignment metadata. - Transform query q to U^T(q-mu), run multi-threaded C++ asymmetric distance table scan, rescore top-R candidates, return top-k IDs and L2/cosine scores, serialize/deserialize binary index files. 11. C++ AND CUDA KERNELS (csrc/) - hadamard_kernels.cu: warp/shared-memory FWHT for 32, 64, 128, 256, 1024 fused with channel scales, outlier sign flips, dither shifts. - trellis_lut_gemm.cu: fused two-multiply 64-byte-table trellis decode+GEMM (1-6 bits); shared-memory 4-bit activation x centroid LUT GEMM (K=8,16,32) fusing min/max, quantize, pack; subspace INT4/INT8 trellis decode+integer dot-product GEMM; single-launch batched multi-adapter residual fusion. - mixed_tile_and_moe_gemm.cu: 2D tile 8/16-bit masked GEMM with local accumulation; nested floor-and-compensation MoE dynamic precision GEMV/GEMM; 24D Golay-coset register decode+GEMM. - kv_and_state_kernels.cu: split-KV E2M1+UE8M0 multi-query speculative verify with causal mask and LSE merge; rotated E8 KV decode keeping tables out of shared memory with single output un-rotation; windowed linear-recurrence update + compensator token + stochastic rounding kernel. - vector_scan.cpp: OpenMP/std::thread scanner decoding 0-12 bit codes via query lookup tables with top-R heap and fine-tier rescoring. - bindings.cpp: pybind11 module exporting all functions with contiguity, dtype, and device checks. 12. STREAMING IO AND CLI (precision_engine/io/container.py, precision_engine/cli.py) - container.py: lazy single-layer streaming reader for sharded binary tensor archives supporting models exceeding single-GPU VRAM; .qpack writer/reader storing schema JSON, bit maps, (alpha,c), sign vectors, key means, compensator tokens, compact adapters, byte-aligned payloads. - cli.py: argparse subcommands quantize-llm, serve-llm, finetune-qat, calibrate-sampling, search-recipe, index-vectors, query-vectors. Deliver complete installable code with zero comments, zero docstrings, zero stubs, zero prohibited features, zero external brand names.
Response not available
Response not available
Response not available