You are an AI evaluation engineer specializing in perceptual...

Prompt

You are an AI evaluation engineer specializing in perceptual quality metrics for generative media. I am upgrading "AI Visual Director Production" v7.8.5 → v8.0.0. Currently, quality gates are PURELY QUALITATIVE (taste ladder, anti-generic prose). Agents cannot make probabilistic routing decisions, compare outputs numerically, or auto-select diagnostics based on measurable failure signals. CURRENT QUALITY SYSTEM: - Rule 32: 7-level taste ladder (Idea → Polish), qualitative - Rule 31: Anti-generic language conversion, binary pass/fail - Rule 35: "Authorize cheapest useful diagnostic" — undefined catalog, no cost model - No numerical thresholds, no automated comparison, no confidence gradients YOUR TASK: Design the complete Quantitative Quality Signals layer AND Diagnostic Decision Matrix. Produce: PART A — QUALITY SIGNALS: 1. Signal catalog with exact metric definitions: CLIP score, aesthetic predictor, drift delta, text-image alignment, audio-visual sync, style consistency, temporal coherence 2. Per-gate threshold table: minimum acceptable values for each signal at each gate 3. Confidence tagging: how to express measurement uncertainty and when to fall back to qualitative assessment 4. Signal provider abstraction: interface contract so any backend (LAION, OpenCLIP, custom) can plug in 5. Composite scoring: how multiple signals combine into a single gate-pass probability PART B — DIAGNOSTIC MATRIX: 1. Complete diagnostic catalog: id, name, cost_tier (LOW/MED/HIGH), applicable_failure_modes[], success_criteria, max_budget, expected_duration 2. Decision tree: mapping (gate_id × failure_type × signal_values) → authorized_diagnostic 3. Escalation protocol: LOW fails twice → MED; MED fails → BLOCK + human escalation; explicit state tracking 4. Budget enforcement: how cumulative diagnostic spend is tracked and capped per gate/project 5. Diagnostic result schema: structured output that feeds back into gate receipt CONSTRAINTS: - Quantitative signals are SUPPLEMENTARY, never sole authority; qualitative override always possible with justification - Thresholds must be empirically grounded (cite typical ranges or define calibration procedure) - Diagnostics must be idempotent and safe to re-run - Matrix must be serializable as JSON for agent consumption - When no quantitative signal is available, system degrades gracefully to v7.8.5 qualitative mode Do NOT ask clarifying questions. Make definitive metric and matrix design choices. Output complete catalogs and decision logic.