All MicroEvals
Build from scratch a complete, self-contained, mixed-modalit...
Create MicroEval

Build from scratch a complete, self-contained, mixed-modalit...

Prompt

Build from scratch a complete, self-contained, mixed-modality quantization, compression, fine-tuning, inference, and vector-retrieval system named precision_engine in Python 3.11+, PyTorch, C++17, and CUDA C++. Implement every file and kernel completely. Exclude everywhere: authentication, authorization, access control, permissions, security policies, auditing, monitoring, compliance telemetry, rate/usage/product limits, quotas, billing, payments, tests, benchmark harnesses, documentation, READMEs, docstrings, comments, placeholders, mocks, dummy code, fake services, stubs, omitted files or functions. Never mention original third-party software, method, vendor, or model brand names; use only the neutral identifiers below. 1. FILES AND BUILD CONFIG Create all 36 files: Root: pyproject.toml, setup.py precision_engine/: __init__.py, cli.py precision_engine/core/: __init__.py, formats.py, transforms.py, trellis_lattice.py precision_engine/calibration/: __init__.py, data_builder.py, scoring.py precision_engine/solvers/: __init__.py, hessian_solvers.py, discrete_solvers.py, annealing_and_transfer.py, specialized_modalities.py precision_engine/training/: __init__.py, qat_and_distill.py, adapters_and_optim.py precision_engine/search/: __init__.py, recipe_controller.py precision_engine/runtime/: __init__.py, kv_and_state.py, moe_and_layers.py, decoding_controller.py, engine.py precision_engine/vector_index/: __init__.py, tiered_index.py precision_engine/io/: __init__.py, container.py csrc/: bindings.cpp, hadamard_kernels.cu, trellis_lut_gemm.cu, mixed_tile_and_moe_gemm.cu, kv_and_state_kernels.cu, vector_scan.cpp pyproject.toml: PEP 517 setuptools/wheel config, deps torch>=2.3.0, numpy>=1.26.0, scipy>=1.11.0, script precision-engine = precision_engine.cli:main. setup.py: CUDAExtension precision_engine._C from all 6 csrc/ files with C++17 -O3, NVCC -O3 --use_fast_math, sm_80/sm_89/sm_90. Each __init__.py exports package symbols. Zero comments. 2. FORMATS AND GRIDS (precision_engine/core/formats.py) Bit-exact pack, unpack, quantize, dequantize: - Primitives: symmetric/asymmetric INT1-INT8, ternary {-1,0,+1}, E2M1 with subnormals, E4M3, E5M2, block scales UE8M0 (power-of-two), UE5M3, signed E4M3. - CompandingOffsetGrid: closed-form 2^b-level grid with companding strength alpha in [0,1] (uniform spacing at 0 to float tail spacing at 1) and asymmetry offset c in [-1,1]. Reuse unused sign bit of E4M3 scale to select G(alpha,c) vs mirror -G(alpha,c), doubling distinct values per block at zero bit cost. fit_companding_offset_grid fits (alpha,c) for 2-6 bits via grid search plus Nelder-Mead. - LogBaseQuantizer: q(x) = sign(x) s b^clamp(round(log_b(max(|x|/s,eps))), -2^(b-1), 2^(b-1)-1), learning base b>1 per tensor/channel for heavy-tailed weights. - ZeroCal4BitFormat: calibration-free layerwise 4-bit quantization over size-64 row sub-blocks via quantile/norm clipping search, packing 4-bit indices with double-quantized scales. - CausalBoundedQuantizer: deterministic causal quantizer keeping synchronized adaptive tables C_t on encoder/decoder (init >=eps spacing or uniform 2 eps step), guaranteeing |x_t - x_hat_t| <= eps from x_<=t alone and updating C_(t+1) strictly from transmitted symbols. 3. TRANSFORMS (precision_engine/core/transforms.py) PyTorch ops and precision_engine._C dispatchers: - fwht, block_hadamard: normalized O(d log d) Fast Walsh-Hadamard Transform for blocks 32, 64, 128, 256, 1024; relocate_subspace_hadamard runs inverse H_128 on activations so weights stay exact integers. - DitheredRHT: sparse random diagonal signs D, Hadamard H, calibrated non-uniform pre-rounding dither, matching subtraction on reconstruction. - OutlierSignAdjustedRotation: single pass finds fixed massive outlier channels (especially MLP output projection inputs post-nonlinearity), builds sign vector s in {-1,+1}^d unifying target Hadamard row signs to collapse bimodal distributions into unimodal profiles, pairs with asymmetric activation quantization absorbing shifted mean via zero-point, applying only element-wise sign flips before fwht at inference. - CayleyOrthogonalRotation: skew-symmetric A = P - P^T, orthogonal Q = (I-A)(I+A)^-1 plus positive channel scales on query, key, value, gate, up projections, optimized on activations. - PCASubspaceRotation: activation SVD basis V, rotate activations, keep top-k coordinates in FP16/BF16, quantize rest to 4 bits, fuse inverse V offline into weights. - LayerLocalTransform: per-channel balance scales and block-32 Hadamard inside each linear layer, absorbing inverses into weights with input hooks and zero cross-layer propagation. 4. TRELLIS AND LATTICE CODECS (precision_engine/core/trellis_lattice.py) - ComputedTrellisCodec: single shift-register decoder for 1-6 bits/element. Multiplies state by two integer coefficients, sums top bits of both products into non-uniform 6-bit index over a 64-byte table whose rare ends store tail magnitudes and dense center covers core; includes INT/FP presets; encode_trellis_viterbi minimizes Hessian error. - SubspaceIntTrellisCodec: Viterbi matching on 128x2 tiles (256 weights) aligned with H_128 constraining outputs to exact INT4/INT8 codewords for integer GEMM. - PolarE8VectorQuantizer: precondition 8D blocks via random signs and normalized Hadamard, split into magnitude (1D Lloyd-Max) and unit direction, build codebook from normalized E8 shells on S^7 (augmenting from max-error blocks if <2^b) indexed by leader, permutation rank, signs; proxy-Hessian H_cc ||W_[:,c]||_2^2 keeps top 1% sensitive channels at high precision. - Lattice24Quantizer: encode 24D blocks via binary Golay coset nearest-point decoding on 24D Leech lattice (~2.7 bpp overall without large codebooks). - MicroScaledTrellisQuantizer: beam-search trellis encoder plus scale-conditioned hash from block-normalized Gaussian CDF outputting exact E2M1 or UE5M3 elements. 5. CALIBRATION AND SCORING (precision_engine/calibration/) - data_builder.py: - build_fragility_weighted_mixture: 2-bit retention scan across task families, rank fragile tasks by low retention, blend fragile samples with generic corpus tokens. - build_reasoning_opening_mixture: generate initial 256 tokens from full-precision model on reasoning prompts (capturing ~20% higher activation variance across initial ~30 steps), mix 9:1 with generic text, pad/truncate to 512 tokens. - scoring.py: - compute_hidden_kl_divergence: mean and 99th-percentile forward KL and top-1 agreement vs full-precision logits on held-out sequences. - detect_repetition_loops: sample long continuations and flag rolling n-gram loops. - compute_gradient_sensitivity: teacher-forced forward/backward passes over tail tokens computing (dL/dy)^2 (y_hat - y)^2 per module. 6. SOLVERS (precision_engine/solvers/) - hessian_solvers.py: - MismatchCorrectedSuccessiveSolver: H = 2 X X^T + lambda I and mismatch X_fp - X_q, orthogonal projection onto weight subspace for closed-form layer-adaptive interpolation, Cholesky inverse, greedy successive rounding or K-beam search on triangular least-squares. - JointScaleIntegerSolver: row warm-start across scale-refitted candidates, +/-1 integer flips recomputing closed-form optimal group scale coupled across row groups via full H, periodic joint row scale solve, diagonal Tikhonov regularization. - FlipRefinementStreamingSolver: lazy-load one linear layer at a time, optimize Hessian-weighted shared block scales, greedy coordinate descent flipping codes to adjacent E2M1 levels minimizing dw^T H dw. - discrete_solvers.py: - GumbelSoftmaxGridSolver: annealed Gumbel-Softmax relaxation over local-shift indices (3-8 levels) jointly optimizing logits and symmetric group scales without zero-points, hardening and projecting back into packed binary formats. - AdaptiveClusterLUTSolver: allocate K in {8,16,32} centroids per matrix via first-order importance under LUT latency budget, K-means init, freeze centroids, train low-rank soft assignment updates annealed to hard indices. - OutputKroneckerFisherSolver: rank-one Kronecker output Fisher approximation jointly optimizing ranges, transforms, clipping, and rounding. - annealing_and_transfer.py: - SimulatedAnnealingDecompositionSolver: calibration-free W approx Q + U V^T + S (rank-32 SVD plus 1% row-sparse S) over 300 geometric cooling steps. - WeightSpaceQuantVectorTransfer: v_q = theta_donor_qat - theta_donor_base, patch theta_target + lambda v_q before PTQ. - specialized_modalities.py: - Tile2DMixedPrecisionAllocator: hardware-aligned 2D tiles (16x16 or 32x32), score via activations or gradient sensitivity, keep top 1%-20% tiles at 16-bit and rest at 8-bit. - MoEPruneAndQuantizeAllocator: joint expert pruning and bit allocation under tight budgets keeping routers, shared experts, and attention in BF16, plus equivalent RMSNorm scale tuning preventing routing drift. - MonocularDepthQuantizer: joint Q/K attention-preserving calibration matching full-precision softmax maps plus channel affine alignment folded into weights. - WorldModelRolloutQuantizer: keep early encoder blocks and control channels in high precision (padded GEMM plus control buffer), calibrating encoder and per-step predictor scales jointly over rollouts. - ImageCompressionJointQuantizer: joint search of channel-balancing exponents and sequential Hessian weight updates under rate-distortion loss. - VariableRateSpeechCodecModulator: segment speech frames via latent variation density and modulate active codebook depth per segment. 7. TRAINING, ADAPTERS, AND OPTIMIZER (precision_engine/training/) - qat_and_distill.py: - QuantizationAwareDistillationTrainer: frozen teacher training quantized student via top-500 forward KL and deployment-informed one-hot cross-entropy on teacher traces. - ProgressiveMultimodalQATTrainer: teacher response self-relabeling, per-layer vision-sensitivity tau, progressive fake-quantization starting on vision-sensitive layers before early text layers. - adapters_and_optim.py: - CompactCoefficientAdapter, CompactAdapterLayer: residual adapter storing ~2.9M coefficients (~11 MB at 27B) modulating shared projections; kept unmerged in GPU memory with no-adapter slot, per-token routing buffer, and chunked workspace for single-kernel mixed-adapter batching. - ResidualSignSubspaceOptimizer: torch.optim.Optimizer subclass storing one BF16 momentum per 2D weight, updating top-r r-by-r singular subspace via AdamW moments and normalized sign updates from orthogonal residual outside subspace. 8. RECIPE SEARCH (precision_engine/search/recipe_controller.py) - ClosedLoopRecipeOptimizer: iterative search over transforms, solvers, grids, bit widths, group sizes, scale precisions (format-aware scale search and distillation), preservation masks, and calibration mixtures, scored by hidden KL and loop detection. - BitTieredAutoCalibrator: sweep cost = w_bits bits + w_rerank rerank_depth with w_bits > w_rerank (rerank_max - rerank_min) evaluating preconditioning off/on inside each tier before escalating bits. 9. RUNTIME AND DECODING (precision_engine/runtime/) - kv_and_state.py: - CenteredMicroScaledKVCache: subtract per-layer/head mean key from initial 256 prefill tokens before rotation/quantization; execute (a) rotated E8 split-KV flash decoding with rotated-domain V accumulation and single output un-rotation, (b) microscaled trellis KV decode, (c) E2M1+UE8M0 fused multi-query speculative verify across K draft queries and GQA groups with LSE merge. - WindowedRecurrentStateManager: quantized state S plus W-token high-precision update buffer for linear-recurrence layers, offload outlier rows/cols into high-precision compensator tokens sharing update paths, smooth residuals, value-axis grouping plus decay-aware mixed precision, stochastic rounding at window steps. - moe_and_layers.py: - NestedPrecisionMoELayer: pack experts in nested floor-and-compensation bit planes, dynamically choosing per-expert precision from router gating scores at runtime. - decoding_controller.py: - GlobalDivergenceSamplingCalibrator: label-free global (T_q, p_q) fit per requested temperature minimizing forward KL, reverse KL, or JSD vs full precision (zero inference overhead). - PerStepAdaptiveSamplingHead: two-layer MLP mapping hidden states and entropy at step t to token-level (T_t, p_t). - engine.py: autoregressive executor integrating transformers, MoE, linear-recurrence hybrids, mixed-adapter routing, speculative verify, adaptive decoding. 10. VECTOR INDEX (precision_engine/vector_index/tiered_index.py) - Subtract centroid mu, learn PCA basis U ordered by descending variance sigma_j^2, greedily allocate b_j in {0..12} bits per coordinate maximizing MSE reduction per bit, encode optional fine-tier residuals, pack variable-bit codes with alignment metadata. - Transform query q to U^T(q-mu), run multi-threaded C++ asymmetric distance table scan, rescore top-R candidates, return top-k IDs and L2/cosine scores, serialize/deserialize binary index files. 11. C++ AND CUDA KERNELS (csrc/) - hadamard_kernels.cu: warp/shared-memory FWHT for 32, 64, 128, 256, 1024 fused with channel scales, outlier sign flips, dither shifts. - trellis_lut_gemm.cu: fused two-multiply 64-byte-table trellis decode+GEMM (1-6 bits); shared-memory 4-bit activation x centroid LUT GEMM (K=8,16,32) fusing min/max, quantize, pack; subspace INT4/INT8 trellis decode+integer dot-product GEMM; single-launch batched multi-adapter residual fusion. - mixed_tile_and_moe_gemm.cu: 2D tile 8/16-bit masked GEMM with local accumulation; nested floor-and-compensation MoE dynamic precision GEMV/GEMM; 24D Golay-coset register decode+GEMM. - kv_and_state_kernels.cu: split-KV E2M1+UE8M0 multi-query speculative verify with causal mask and LSE merge; rotated E8 KV decode keeping tables out of shared memory with single output un-rotation; windowed linear-recurrence update + compensator token + stochastic rounding kernel. - vector_scan.cpp: OpenMP/std::thread scanner decoding 0-12 bit codes via query lookup tables with top-R heap and fine-tier rescoring. - bindings.cpp: pybind11 module exporting all functions with contiguity, dtype, and device checks. 12. STREAMING IO AND CLI (precision_engine/io/container.py, precision_engine/cli.py) - container.py: lazy single-layer streaming reader for sharded binary tensor archives supporting models exceeding single-GPU VRAM; .qpack writer/reader storing schema JSON, bit maps, (alpha,c), sign vectors, key means, compensator tokens, compact adapters, byte-aligned payloads. - cli.py: argparse subcommands quantize-llm, serve-llm, finetune-qat, calibrate-sampling, search-recipe, index-vectors, query-vectors. Deliver complete installable code with zero comments, zero docstrings, zero stubs, zero prohibited features, zero external brand names.

Response not available

Drag to resize

Response not available

Drag to resize

Response not available

Drag to resize
Drag to resize
Drag to resize