Designing a 1M+ RPS Write-Heavy API Platform in Go
Prompt
Design a production-grade write-heavy API platform in Go that can sustain 1,000,000+ requests per second at peak load. The system is primarily CRUD-based but optimized for extremely high write throughput. Requirements: - API layer written in Go - HTTP/gRPC based service design - Multi-region deployment - Low latency requirements: - p99 < 100ms for API acknowledgement - eventual consistency is acceptable for non-critical reads - Write-heavy workload: - 90% writes - 10% reads - Average request payload: 2KB - Peak traffic: 1M requests/sec - Data retention: 5 years - Expected data growth: billions of records Design: 1. High-level architecture diagram showing: - Load balancers - API gateways - Go services - Databases - Caches - Message queues/streams - Background workers - Observability stack - External dependencies 2. Explain request flow: - From client request arrival - Authentication - Validation - Write processing - Queueing - Persistence - Cache updates - Async processing 3. Choose and justify technologies: - Database choice (PostgreSQL, CockroachDB, Cassandra, DynamoDB, etc.) - Cache layer (Redis, Memcached, etc.) - Queue/streaming system (Kafka, NATS, Pulsar, etc.) - Load balancing strategy - Storage strategy 4. Go implementation considerations: - HTTP server framework - Goroutine/concurrency model - Connection pooling - Backpressure handling - Rate limiting - Circuit breakers - Graceful shutdown - Memory optimization 5. Database design: - Schema examples - Partitioning strategy - Sharding approach - Indexing strategy - Handling hot keys - Handling massive write throughput 6. Reliability: - Failure scenarios: - Database outage - Queue backlog - Cache failure - Region failure - Network partitions - Recovery strategies 7. Scaling strategy: - How the system scales from 100K RPS β 1M RPS β 10M RPS - Bottleneck identification - Horizontal scaling approach 8. Benchmark plan: Provide a realistic load testing strategy using tools like: - k6 - Go benchmarks - Vegeta - Gatling Include: - Expected throughput numbers - CPU/memory requirements - Database write capacity assumptions - Network bandwidth calculations - Latency measurements - Failure testing methodology Avoid generic explanations. Make concrete engineering decisions and defend tradeoffs. Assume the system must run on self-managed infrastructure with a fixed monthly budget. Do not solve scaling problems by infinitely adding servers.
