When 200 requests per second is loaded, around 2% to 10% of the times, HTTP status 502 returns and around 1% it is 504 as well.
We run Play server in K8s cluster in private cloud with 3 replicas. K8s configured with ingress nginx proxy server that directs the request to Play. We use OpenJDK JRE 1.8.
The time out setting are made as below.
play.server.http.idleTimeout = 180s
play.server.akka.requestTimeout = 300s
Tried both default server and Netty to see same issue. This issue doesnot occur even if we load 1000 concurrent requests in development VM. We figured out that the private cloud provider network is pretty slow and that is causing this issue. Now, how do we customise the configuration of Play for such slow network scenario so that we can reduce the error rate in Jmeter load testing?
Play server log reports no error. However, the nginx proxy server log shows below error:
recv() failed (104: Connection reset by peer) while reading response header from upstream.
Any suggestion or help is appreciated.