Universal AI Real-World Capability Benchmark

Prompt

# Universal AI Capability Benchmark — Real-World Performance Test You are being evaluated as a general-purpose AI assistant on your ability to solve realistic problems accurately, intelligently, efficiently, and practically. This is an evaluation, not a conversation. Your response will be compared against other AI models. Optimize for **correctness, reasoning quality, instruction following, factual reliability, practical usefulness, clarity, and completeness**. ## GENERAL RULES 1. Follow every instruction exactly. 2. Answer the actual question rather than giving generic advice. 3. Do not invent facts, sources, APIs, libraries, commands, statistics, or capabilities. 4. If information is uncertain, explicitly state the uncertainty. 5. Separate: * Facts * Assumptions * Reasoning * Recommendations 6. Prefer concrete examples over vague explanations. 7. For technical questions, provide technically executable solutions. 8. For code, prioritize correctness, security, maintainability, and compatibility. 9. If the problem is ambiguous, identify the ambiguity and make the most reasonable assumption. 10. Do not unnecessarily ask follow-up questions when the problem can reasonably be solved with assumptions. 11. Do not blindly agree with the user. Correct false assumptions when necessary. 12. Consider edge cases and failure modes. 13. When comparing alternatives, explain the trade-offs rather than declaring a winner without justification. 14. Do not use unnecessary verbosity. Be comprehensive when the task requires it. 15. Never claim that you performed an action, accessed a system, executed code, browsed the internet, or verified information unless you actually did so. --- # TASK 1 — REAL-WORLD PROBLEM SOLVING A user wants to build a production-grade AI-powered application for managing personal finances. The application should eventually support: * User authentication * Personal financial accounts * Transactions * Budgets * Financial goals * Receipt/document uploads * AI-powered transaction categorization * Financial insights * Reports * Forecasting * Investment tracking * Notifications * Search * Mobile/PWA access * Multi-user access * Data privacy * Auditability The first version must be realistic for a small engineering team. ### Your task Create a practical plan for turning this idea into a production MVP. Explain: 1. What should be included in V1. 2. What should explicitly NOT be included in V1. 3. Recommended architecture. 4. Recommended technology stack. 5. Database design. 6. Authentication strategy. 7. AI architecture. 8. Security architecture. 9. API design. 10. Testing strategy. 11. Deployment strategy. 12. Monitoring and observability. 13. Development roadmap. 14. Biggest technical risks. 15. Biggest product risks. Do not over-engineer the MVP. --- # TASK 2 — INSTRUCTION FOLLOWING Follow these instructions exactly. Write a response containing exactly **7 sections**. The sections must be named: 1. Problem 2. Assumptions 3. Solution 4. Architecture 5. Risks 6. Roadmap 7. Final Recommendation Additional requirements: * The "Problem" section must contain exactly 3 bullet points. * The "Assumptions" section must contain exactly 4 bullet points. * The "Risks" section must contain exactly 5 risks. * The "Roadmap" section must contain exactly 4 phases. * Include exactly one comparison table. * The table must contain exactly 4 columns. * Do not use emojis. * Do not include a conclusion outside section 7. --- # TASK 3 — KNOWLEDGE AND FACTUALITY Answer the following questions. ### A. What is the difference between: * Machine Learning * Deep Learning * Generative AI * Large Language Models * AI Agents Explain the relationship between them without incorrectly treating them as interchangeable terms. ### B. Explain the difference between: * SQL database * NoSQL database * Vector database * Object storage * Cache Give one appropriate use case for each. ### C. Explain the difference between: * Authentication * Authorization * Encryption * Hashing * Digital signatures Provide a concrete example for each. ### D. Explain why increasing model parameters does not automatically guarantee better real-world performance. Do not give a simplistic "larger is always better" or "smaller is always better" answer. --- # TASK 4 — CODING Consider this Python function: ```python def find_duplicates(items): duplicates = [] for item in items: if items.count(item) > 1: duplicates.append(item) return duplicates ``` ### Requirements 1. Identify all important problems with this implementation. 2. Explain its time complexity. 3. Rewrite it efficiently. 4. Preserve the original input. 5. Return each duplicated value only once. 6. Explain the new time and space complexity. 7. Include test cases for: * Empty input * No duplicates * Multiple duplicates * All values duplicated * A value appearing three times * Strings * Integers Do not use external libraries. --- # TASK 5 — CODE DEBUGGING The following JavaScript code is intended to fetch users and display their names: ```javascript async function loadUsers() { const response = fetch("/api/users"); const users = response.json(); document.querySelector("#users").innerHTML = users.map(user => `<li>${user.name}</li>`).join(""); } ``` ### Your task 1. Identify every bug or important issue. 2. Explain why each issue occurs. 3. Provide corrected code. 4. Handle HTTP errors. 5. Handle network errors. 6. Prevent unsafe HTML injection. 7. Explain whether additional production improvements are needed. --- # TASK 6 — SOFTWARE ARCHITECTURE Design an architecture for an AI application that can: * Accept user questions. * Retrieve information from private documents. * Call external tools. * Use multiple AI models. * Maintain conversation history. * Perform structured reasoning. * Retry failed operations. * Detect model failures. * Log important events. * Evaluate answer quality. Compare these approaches: A. Single LLM application B. RAG application C. Tool-using AI agent D. Multi-agent system E. Workflow-based AI system Explain: * When each architecture is appropriate. * Advantages. * Disadvantages. * Complexity. * Reliability. * Cost. * Observability. * Security implications. Then recommend the architecture for a production MVP and explain why. --- # TASK 7 — REASONING A software company has three engineering teams. Team A can complete a feature in 8 days. Team B can complete it in 12 days. Team C can complete it in 24 days. Assume their work rates are independent and additive. Answer: 1. What fraction of the feature does each team complete per day? 2. How long would they theoretically take working together? 3. Explain the calculation. 4. Identify at least two real-world assumptions that make this theoretical calculation unrealistic. Do not only provide the final number. --- # TASK 8 — DATA INTERPRETATION Consider this fictional dataset: | Model | Accuracy | Latency | Cost / 1M tokens | | ----- | -------: | ------: | ---------------: | | A | 91% | 1.2s | $2 | | B | 94% | 2.8s | $8 | | C | 92% | 0.8s | $1 | | D | 96% | 5.1s | $15 | Answer: 1. Which model has the highest accuracy? 2. Which has the lowest latency? 3. Which is cheapest? 4. Which model would you choose for a latency-sensitive consumer application? 5. Which would you choose for a high-stakes reasoning workload? 6. Explain why there is no universally best model. 7. Identify the trade-offs. Do not invent information that is not present in the table. --- # TASK 9 — QA / CRITICAL THINKING Evaluate this claim: > "AI will replace software engineers because AI can already generate code." Determine whether the claim is: * True * Mostly true * Partially true * Mostly false * False Then explain your answer. Your analysis must distinguish between: * Code generation * Software engineering * System design * Requirements engineering * Debugging * Testing * Security * Deployment * Maintenance * Product decisions * Human responsibility Avoid both extreme optimism and extreme pessimism. --- # TASK 10 — RESEARCH QUALITY Suppose a user asks: > "What is the best AI model in the world?" Explain why this question is poorly defined. Then identify the dimensions that should be used to answer it, including: * Reasoning * Coding * Mathematics * Knowledge * Long-context performance * Tool use * Agentic performance * Multimodal capability * Latency * Cost * Reliability * Safety * Real-world task performance Explain why benchmark scores alone may not identify the best model for a particular user. --- # TASK 11 — DECISION MAKING You have three possible engineering approaches: ### Option A Fast to build, inexpensive, but difficult to scale. ### Option B Moderate development time, moderate cost, strong scalability. ### Option C Very expensive and complex, but potentially the highest long-term scalability. A startup has: * 2 developers * Limited funding * 5,000 expected users in year one * Uncertain product-market fit * A deadline of 10 weeks Which option should they choose? Explain the decision using: * Risk * Time * Cost * Scalability * Reversibility * Opportunity cost * Product uncertainty Do not automatically choose the most technically sophisticated option. --- # TASK 12 — SECURITY Identify the major security risks in an AI application that: * Stores private user documents. * Sends document content to an LLM. * Uses third-party APIs. * Supports tool calling. * Stores conversation history. * Allows file uploads. Explain the risks and mitigations for: * Prompt injection * Data leakage * Excessive permissions * Malicious files * SSRF * Credential exposure * Cross-tenant data access * Insecure tool calls * Logging sensitive information * Model-generated unsafe actions Prioritize the risks rather than simply listing them. --- # TASK 13 — PRODUCT THINKING A team wants to build an AI application with 30 features. They have 8 weeks and 3 developers. Explain how you would decide which features belong in the MVP. Provide: 1. A prioritization framework. 2. Example scoring criteria. 3. A sample feature prioritization. 4. Features you would postpone. 5. How user feedback should influence the roadmap. --- # TASK 14 — COMMUNICATION Explain the concept of "technical debt" to: 1. A beginner programmer. 2. A CTO. 3. A non-technical startup founder. The underlying concept must remain technically accurate while the explanation changes for each audience. --- # TASK 15 — FAILURE ANALYSIS An AI-powered production application suddenly experiences: * 3× latency. * 20% API failures. * Increased hallucinations. * Higher database CPU usage. * Increased user complaints. You are the lead engineer. Explain your response in chronological order. Include: 1. Immediate actions. 2. Incident investigation. 3. Hypotheses. 4. Metrics/logs to inspect. 5. Mitigation. 6. Root-cause analysis. 7. Long-term prevention. 8. Post-incident review. Do not assume the root cause before investigating. --- # FINAL EVALUATION After completing all tasks, provide a final self-assessment. Score yourself from 0–10 on: | Category | Score | | --------------------- | ----: | | Instruction Following | | | Factual Accuracy | | | Reasoning | | | Coding | | | Debugging | | | Architecture | | | Security | | | Product Thinking | | | Communication | | | Practicality | | For every score, provide a one-sentence justification. Then provide: ### Strongest Areas List the 3 areas where your response is strongest. ### Weakest Areas List the 3 areas where your response is weakest. ### Potential Errors Identify any statements or recommendations in your response that may require external verification. ### Final Verdict Give a concise assessment of how reliable this response would be for a real-world software engineering team. Do not claim that this self-assessment is objective or equivalent to an independent evaluation.