Using more threads in play framework with akka http server

We have a web server that is running in production from past year and have no issues but recently there is a surge in the user base and since then we see severe slowness in the business hours and we are sometime seeing server going unresponsive as well so this is a severe problem for us.

We observe the max utilization in the DB side is at max 8 active connections and no more. So, we have narrowed down the problem to ‘Play framework or the akka server is allowing only a limited thread pool to manage all the load and thus causing the bottleneck’.

Being fairly new to the play framework and akka http server, we have tried the below configs in various combinations but still not seeing increased thread pool in the run time. It is always 8.
Note: This ‘8’ is seen consistent on a pc with 8 logical cores and on our production server that has 6 cores.

Thread name sample for reference : application-akka.actor.default-dispatcher-3

Sample config below:

#akka.action.default-dispatcher.executor = "fork-join-executor"
akka.action.default-dispatcher.executor = "thread-pool-executor"
#akka.action.default-dispatcher.throughput = 100
akka.action.default-dispatcher.thread-pool-executor.fixed-pool-size = 300
#akka.action.default-dispatcher.thread-pool-executor.core-pool-size-min = 2
#akka.action.default-dispatcher.thread-pool-executor.core-pool-size-factor = 2.0
#akka.action.default-dispatcher.thread-pool-executor.core-pool-size-max = 10
#akka.action.default-dispatcher.fork-join-executor.parallelism-max = 24

akka.http.host-connection-pool.max-connections = 20

According to the docs here, it should be akka.actor.default-dispatcher instead of akka.action.default-dispatcher.

Hi @ck,

:+1:

Akka and Akka HTTP being asynchronous frameworks can work with a low number of threads. You just need to make sure not to block on those threads.

You can try to use jstack <pid> on the command line to find out where threads are blocking and whether that’s an issue (you can also use the commercial Akka Thread Starvation Detector • Akka Enhancements in case you have a Lightbend subscription for more automatic analysis).

Another reason might be that the DB connection pool is limited by something else.

Johannes