POC

Prompt

I want you to act as a senior AI/ML architect, data engineer, and enterprise commercial-intelligence architect. I am building an internal AI-powered Commercial & Sales Intelligence MVP for my company. I want you to critically review our current direction before we invest more time in implementation. IMPORTANT: Do not simply agree with my approach. Do not praise the project. Do not assume my architecture is correct. Challenge my assumptions, identify weaknesses, unnecessary complexity, missing components, data risks, scalability problems, and bad architectural decisions. ================================================== BUSINESS OBJECTIVE ================================================== The goal is to build an internal commercial intelligence system that helps our sales and management teams learn from historical projects and quotations. The system should eventually help answer questions such as: - What happened in previous projects? - Why did we win or lose projects? - Have we seen a project similar to this opportunity before? - What characteristics appear repeatedly in successful/lost projects? - What information is missing before preparing a quotation? - What lessons from previous projects should Sales investigate? - What commercial patterns exist in our historical data? - What should management investigate before making a commercial decision? The system must NOT automatically decide whether we should bid or reject a project. The human commercial team remains responsible for the decision. The system should provide evidence, historical context, analytics, similar projects, risks/signals, unknowns, and questions. ================================================== CURRENT DEVELOPMENT STAGE ================================================== We first created a synthetic dataset of 30 projects to validate the architecture. The synthetic data was intentionally designed with hidden patterns so we could test whether the analytics pipeline could recover known relationships. The synthetic dataset is NOT considered real business intelligence. Management has now approved an MVP pilot using REAL Eastern Zone company data. The Eastern Zone team includes: - Area Manager / Sales - Senior Sales - Other sales employees - Engineers - Documentation staff - Medical sales employee The next stage is to collect and understand real historical data from the Eastern Zone. ================================================== CURRENT TECHNICAL DIRECTION ================================================== Frontend: - Angular 21 - PrimeNG - Tailwind CSS Backend: - Python - FastAPI Database: - PostgreSQL - pgvector for semantic/vector search Analytics: - Pandas - Scikit-learn - XGBoost Local AI: - Ollama - Qwen family models Current development hardware: - RTX 3070 8GB VRAM - 32GB RAM The current Qwen model we tested successfully is Qwen3 8B. We are also considering newer/larger Qwen models and do NOT want the architecture tightly coupled to one model. Future infrastructure: - NVIDIA DGX Spark - larger local AI models - company-controlled AI infrastructure - PostgreSQL/pgvector - RAG - internal AI services - potentially multiple company departments and regions Network/security concept: Users/laptops → secure company network/VPN/Tailscale → internal AI server → database/AI services. We do NOT want to expose the AI server directly to the public internet. ================================================== CURRENT AI ARCHITECTURE ================================================== Our current conceptual architecture is: User ↓ Angular Dashboard ↓ FastAPI ↓ Business Logic ↓ Evidence / Analytics Layer ↓ PostgreSQL + pgvector ↓ Analytics / Similarity / ML ↓ Qwen ↓ Human-readable explanation ↓ Human decision We want the LLM to explain evidence rather than invent evidence. ================================================== EVIDENCE-FIRST DESIGN ================================================== We currently distinguish: FACT = directly supported by documented company data. INFERENCE = a conclusion derived from documented evidence. UNKNOWN = information that is not available or documented. Example: FACT: "Healthcare had 8 wins from 11 historical opportunities." INFERENCE: "Healthcare had a substantially higher historical win rate than the overall sample." UNKNOWN: "Competitor pricing is not available." The LLM should never turn UNKNOWN into a FACT. Python/SQL should calculate deterministic metrics. The LLM should primarily interpret, summarize, explain, and answer questions using retrieved evidence. ================================================== CURRENT ANALYTICS ================================================== The system currently calculates things such as: - Overall win rate - Win/loss counts - Win rate by sector - Win rate by city - Win rate by lead source - Win rate by project size - Win rate by delivery requirement - Win rate by technical compliance - Margin patterns - Discount patterns - Loss reasons - Similar historical projects We also experimented with XGBoost to identify variables that the model relied on when distinguishing historical wins/losses. We understand that: - 30 projects are far too few for serious predictive modeling. - The synthetic dataset is only an architecture/analytics validation tool. - Feature importance does not establish causality. - An experimental model output must not automatically be presented as a probability. - Real data needs proper validation and time-aware evaluation. ================================================== SIMILAR PROJECT ENGINE ================================================== For a new quotation, the system searches historical projects using business and commercial similarity. For example: New opportunity: Healthcare Riyadh Consultant SAR 2.4M 45-day delivery 18% expected margin 6% discount Full technical compliance The system returns historically similar projects and shows: - project details - similarity score - outcome - loss reason where available We want the similarity engine to help Sales understand: "What happened in previous projects that looked somewhat like this?" It should NOT imply that similar projects guarantee the same outcome. ================================================== CURRENT AI OUTPUT STRUCTURE ================================================== The intended Qwen response structure is: 1. Executive Summary 2. Facts From the Data 3. Positive Historical Signals 4. Negative Historical Signals 5. Conflicting Signals 6. Similar Historical Projects 7. Inferences 8. Unknown Information 9. Questions for Sales 10. Data Limitations 11. Human Decision Required The evidence package contains: - facts - conflicts - similar projects - ML signals - unknowns - limitations The LLM receives this evidence rather than directly calculating everything from raw data. ================================================== DATA MODEL DIRECTION ================================================== Historical project fields may include: - Project ID - Project Name - Client - Sector - City - Project Type - Salesperson - Lead Source - Quotation Date - Estimated Project Value - Quoted Value - Final Award Value - Estimated Cost - Expected Margin % - Discount % - Required Delivery Days - Payment Terms - Outcome - Loss Reason - Competitor - Competitor Price - Technical Compliance - Client Feedback - Actual Cost - Actual Margin % - Actual Delivery - Project Issues - Lessons Learned We understand that not all historical records will contain all fields. Missing information itself may be valuable because it shows what the company does not currently capture. We want controlled values such as: - Known - Unknown - Not Applicable rather than using fake values such as zero. ================================================== COMPANY KNOWLEDGE ================================================== We also plan to maintain structured company knowledge covering: - Company overview - Products - Services - Capabilities - Regions - Terminology - Commercial rules - What the company can and cannot do This company knowledge should be separated from historical transaction data. ================================================== IMPORTANT PRINCIPLES ================================================== We want: 1. Evidence before AI explanation. 2. SQL/Python for deterministic calculations. 3. LLM for interpretation and explanation. 4. Human approval for important decisions. 5. Clear distinction between fact, inference, and unknown. 6. Traceability/provenance. 7. Auditability. 8. Model-agnostic architecture. 9. Real data separated from synthetic data. 10. Small MVP first, then scale. 11. No unnecessary multi-agent complexity. 12. No premature fine-tuning. 13. No automatic bid/no-bid decisions. 14. No invented company information. 15. No pretending small historical samples are statistically reliable. ================================================== FUTURE VISION ================================================== If the Eastern Zone MVP demonstrates real value, the company may eventually deploy the system on dedicated infrastructure such as DGX Spark. The future platform could support: - larger local models - RAG - document understanding - quotation analysis - commercial intelligence - multiple regions - more users - more historical data - internal company knowledge - potentially public opportunity research - continuous improvement But we do NOT want to over-engineer the MVP for this future vision. ================================================== WHAT I WANT YOU TO DO ================================================== Now critically audit this entire approach. I want you to answer the following: 1. ARCHITECTURE REVIEW Is the overall architecture technically sound? What would you change? 2. DATA ARCHITECTURE Is PostgreSQL + pgvector appropriate? What entities/tables are missing? What relationships should exist? 3. AI ARCHITECTURE Is the separation between deterministic analytics, ML, RAG, and LLM correct? Where should each technology be used? Where should it NOT be used? 4. EVIDENCE ARCHITECTURE Is FACT / INFERENCE / UNKNOWN enough? What additional controls would you add to prevent hallucination or unsupported conclusions? 5. ML Is XGBoost appropriate? When should we actually introduce predictive modeling? What minimum data quality/sample size considerations should we have? How should validation be performed? 6. SIMILARITY Should similarity use: - SQL filters? - classical feature similarity? - embeddings? - pgvector? - hybrid search? Recommend the correct approach and explain why. 7. RAG What exactly should be embedded? What should remain structured SQL data? How should structured and unstructured retrieval work together? 8. QWEN What should the LLM actually be responsible for? What should NEVER be delegated to the LLM? Should we use Qwen3 8B, a newer Qwen model, or keep the model layer interchangeable? 9. DATA COLLECTION We are about to interview the Eastern Zone team and collect historical project information. What information should we prioritize? What data should we avoid collecting because it has low value or creates unnecessary complexity? 10. SECURITY What should we implement now for the MVP versus later for production? Consider: - authentication - authorization - audit logs - secrets - network access - database security - PII - customer information - financial information - model access 11. SCALABILITY Will this architecture migrate cleanly from: RTX 3070 → DGX Spark → potentially larger company infrastructure? What should we design now so we don't have to rebuild later? 12. WHAT WE ARE OVERENGINEERING Identify anything in our current plan that is unnecessary for a 2-month Eastern Zone MVP. 13. WHAT WE ARE UNDERENGINEERING Identify anything we are currently overlooking that could become a serious problem later. 14. BIGGEST RISKS Give me the 10 biggest technical/business risks in this project, ordered by severity. 15. MVP BOUNDARY Define exactly what should be included in the final 2-month MVP and what should explicitly be postponed. 16. RECOMMENDED FINAL ARCHITECTURE Give me the architecture you would personally implement if you were responsible for this project. Show it as a clear diagram. 17. IMPLEMENTATION ORDER Give me the recommended development sequence from Day 1 of real-data collection until the final management demo. 18. SUCCESS CRITERIA Define measurable criteria that would prove the MVP is actually useful to the business. 19. MANAGEMENT TRUST What would make management trust the system? What would make them distrust it? 20. FINAL VERDICT Do NOT give me a generic "looks good." Tell me: - What parts of our direction are correct? - What parts need modification? - What should we stop doing? - What should we start doing? - What should we postpone? - What is the single biggest architectural issue you see? - What is the single biggest business risk? - What should I focus on during the next 2 months? Be critical, practical, and specific. Assume that I am technically capable and want an architecture that can eventually become a real internal company platform, but I do NOT want to over-engineer the MVP.