Most engineering groups depend on auto-scaling to avoid wasting them. But on the “Super Bowl standard” of scale, auto-scaling is a lie. It is just too reactive. By the time your cloud supplier spins up new situations, your latency has already spiked, your database connection pool is exhausted and your customers are looking at a 500 error.
Here are the 4 architectural patterns we use to outlive large concurrency. These apply whether or not you might be streaming touchdowns or processing checkout queues for a limited-edition sneaker drop.
1. Aggressive load shedding
The greatest mistake engineers make is attempting to course of each request that hits the load balancer. In a high-concurrency occasion, that is suicide. If your system capability is 100,000 requests per second (RPS) and also you obtain 120,000 RPS, attempting to serve everybody normally leads to the database locking up and zero folks getting served.
