this is a pretty interesting question that could provide enough stuff for a whole article…
I guess the underlying question here is how queuing and timeouts work together for such a spike and what other resources are needed to queue up 100000 requests.
Since each TCP connection can only handle a single request at a time with HTTP/1.1, let’s assume we are talking about 100000 incoming TCP connection in one ms. This will be pretty tough on the OS, so it depends a lot on how your TCP stack is configured. Here is only a very quick rundown on what is involved:
- How many (TCP SYN) packets can the network deliver in 1ms?
- Can the OS process these packets (complete TCP handshake) or will some be dropped?
- If some packets are dropped will they be retried by the client and for how long?
- After the connection is accepted it will be put into a socket backlog, what’s the size of the backlog and how much memory does it need per connection?
- How many memory / resources does the OS need to maintain 100000 TCP connections (buffer sizes, config data in ipfilter etc)
- When the TCP handshake is successful, will the clients wait for long enough after they have sent their requests before timing out?
- Akka HTTP has a max-connections setting which governs how many connections to accept concurrently, when that number is reached, TCP connections should be backlogged in the socket TCP backlog which is maintained by the OS.
If there is more intermediate infrastructure like reverse proxies etc, then these will introduce their own settings and queues and you will have to take them into account as well.
I’d run performance tests, and then try to find out what the current problems are. This is likely more of an OS TCP stack optimization problem than an Akka problem after all.