# Artificial Analysis Microeval Suite: Multimodal Video Type...

Prompt

# Artificial Analysis Microeval Suite: Multimodal Video Typesetting & Translation Goal: Evaluate models for high-accuracy, low-cost video typesetting, scene-context translation, and subtitle alignment. ## Test Case 1: Structured JSON Translation & Glossary Adherence Prompt: "Translate the following array of Japanese text snippets extracted from video OCR frames into English. Apply the following strict constraints: 1. Enforce given-name family-name ordering (e.g., '外薗健' -> 'Takeru Hokazono'). 2. Return ONLY a valid JSON object matching this schema: `{"translations": [{"id": int, "original": str, "translated": str}]}`. 3. Do not alter formatting or add markdown outer blocks. Input: [ {"id": 1, "text": "原作：外薗健"}, {"id": 2, "text": "監督：竹内哲也"}, {"id": 3, "text": "TVアニメ 2027年4月放送開始"} ]" Evaluator Checks: - [ ] Schema Validation: Returns valid JSON without missing keys or extra formatting. - [ ] Translation Accuracy: Correctly translates names using anime industry standards rather than literal dictionary definitions. - [ ] Latency & Cost: Measure Time-to-First-Token (TTFT) and cost per 1,000 requests. --- ## Test Case 2: Multimodal Scene Context & Typesetting Recommendations Prompt: "[Image Input: Anime scene keyframe with text overlay] Analyze the provided video frame along with its OCR bounding box `[ymin, xmin, ymax, xmax]`. 1. Determine the category of text (e.g., 'dialogue_subtitle', 'signboard', 'main_title', 'credit_staff'). 2. Suggest an ASS subtitle style hex color code for maximum contrast against the surrounding background. 3. Output format: `{"category": str, "contrast_color_hex": str, "translation": str}`" Evaluator Checks: - [ ] Visual Context Awareness: Accurately identifies text type based on visual position. - [ ] Color Contrast Logic: Selects high-visibility hex colors that stand out against background pixels.