TechEmpower performance benchmark results round 15, including several play2 setups


#1

Not sure if this is known already, since the benchmarks were published February 14, but here it is: https://www.techempower.com/benchmarks/#section=data-r15&hw=ph&test=json

For those not in the know, this is a massive set of benchmarks for a lot of technologies and frameworks. The filter panel allows you to do fine-grained selection of what’s important to you and then you can check the results, which reach over plenty of tests for json serialization, plain text, queries, etc… both on physical hosts and on cloud hosting(Azure).

It’s certainly an interesting set of data as a whole, and more specifically for play2’s place in it.

Beyond posting this as a purely informative thing, I wonder if there’s been communication between the dev team(or Lightbend) and the people behind these benchmarks, in relation to how the setup of the play instances happened.

I’m reminded of this: https://github.com/playframework/playframework/issues/8010

EDIT: Nevermind, it’s all listed on their github.

For instance, play2-java can be viewed here: https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Java/play2-java

EDIT2: I just stumbled upon this: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Java/play2-java/play2-java-ebean-hikaricp/build.sbt

Where the Netty server is enabled yet the Akka server isn’t disabled, which is stated to be necessary if you’re going to run Netty on 2.6.x: https://www.playframework.com/documentation/2.6.x/NettyServer

To use the Netty server backend you first need to disable the Akka HTTP server and add the Netty server plugin to your project

Not sure if that means Akka is still running the whole time, if there’s performance issues because of that, etc…

Yeah, smarter people than me can probably make a more balanced judgement of this, I suppose. I’ll stop editing the top post now.


(Rich Dougherty) #2

Can you try running one of the projects to see which server is actually running? I think it was @mkurz and @godenji who were kind enough to update the tests in the latest round.


#3

I launched the play2-java project, which sets Netty Server.

Starting it up in dev mode just shows:


--- (Running the application, auto-reloading is enabled) ---

[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

(Server started, use Enter to stop and go back to the console...)

Checking the plugins give this list:

	sbt.plugins.JvmPlugin: enabled in root
	sbt.plugins.CorePlugin: enabled in root
	sbt.plugins.JUnitXmlReportPlugin: enabled in root
	sbt.plugins.Giter8TemplatePlugin: enabled in root
	play.sbt.Play: enabled in root
	play.sbt.PlayAkkaHttp2Support
	play.sbt.PlayAkkaHttpServer: enabled in root
	play.sbt.PlayFilters: enabled in root
	play.sbt.PlayJava: enabled in root
	play.sbt.PlayLayoutPlugin: enabled in root
	play.sbt.PlayLogback: enabled in root
	play.sbt.PlayMinimalJava
	play.sbt.PlayNettyServer: enabled in root
	play.sbt.PlayScala
	play.sbt.PlayService
	play.sbt.routes.RoutesCompiler: enabled in root
	play.sbt.test.MediatorWorkaroundPlugin
	play.twirl.sbt.SbtTwirl: enabled in root
	com.typesafe.sbt.SbtNativePackager: enabled in root
	com.typesafe.sbt.packager.archetypes.JavaAppPackaging: enabled in root
	com.typesafe.sbt.packager.archetypes.JavaServerAppPackaging: enabled in root
	com.typesafe.sbt.packager.archetypes.jar.ClasspathJarPlugin
	com.typesafe.sbt.packager.archetypes.jar.LauncherJarPlugin
	com.typesafe.sbt.packager.archetypes.scripts.AshScriptPlugin
	com.typesafe.sbt.packager.archetypes.scripts.BashStartScriptPlugin: enabled in root
	com.typesafe.sbt.packager.archetypes.scripts.BatStartScriptPlugin: enabled in root
	com.typesafe.sbt.packager.archetypes.systemloader.SystemVPlugin
	com.typesafe.sbt.packager.archetypes.systemloader.SystemdPlugin
	com.typesafe.sbt.packager.archetypes.systemloader.SystemloaderPlugin
	com.typesafe.sbt.packager.archetypes.systemloader.UpstartPlugin
	com.typesafe.sbt.packager.debian.DebianDeployPlugin
	com.typesafe.sbt.packager.debian.DebianPlugin: enabled in root
	com.typesafe.sbt.packager.debian.JDebPackaging
	com.typesafe.sbt.packager.docker.DockerPlugin: enabled in root
	com.typesafe.sbt.packager.docker.DockerSpotifyClientPlugin
	com.typesafe.sbt.packager.jdkpackager.JDKPackagerDeployPlugin
	com.typesafe.sbt.packager.jdkpackager.JDKPackagerPlugin
	com.typesafe.sbt.packager.linux.LinuxPlugin: enabled in root
	com.typesafe.sbt.packager.rpm.RpmDeployPlugin
	com.typesafe.sbt.packager.rpm.RpmPlugin: enabled in root
	com.typesafe.sbt.packager.universal.UniversalDeployPlugin
	com.typesafe.sbt.packager.universal.UniversalPlugin: enabled in root
	com.typesafe.sbt.packager.windows.WindowsDeployPlugin
	com.typesafe.sbt.packager.windows.WindowsPlugin: enabled in root
	com.lightbend.sbt.javaagent.JavaAgent
	com.lightbend.sbt.javaagent.JavaAgentPackaging
	com.typesafe.sbt.web.SbtWeb: enabled in root
	com.typesafe.sbt.jse.SbtJsEngine: enabled in root
	com.typesafe.sbt.jse.SbtJsTask: enabled in root

Which shows both Akka and Netty as “enabled in root”.

So, I’m not sure what to make of that.

Adding .disablePlugins(PlayAkkaHttpServer) in the build.sbt produces the exact same plugin list, still saying the akka http server is enabled in root

Is there another way I should be looking at this?

EDIT: Also the content of the zip produced by sbt dist is identical, with or without akk disabled in build.sbt

EDIT2: Also, there seem to be certain steps missing from the dockerfile in the database projects. They all expect a mysql database yet the code contains no setup details, nor do the dockerfiles.


(Rich Dougherty) #4

If you take a thread dump of the running app, take a look for Netty threads. That should confirm whether it’s running.


(Rich Dougherty) #5

They’re probably connecting to TechEmpower’s provided MySQL database. On the TE benchmark site there are descriptions about how each test should behave.


#6

Sorry for the late reply. I ran the project from the jar produced by sbt stage, as per the dockerfile.

I ran it 3 times, each time I called the routes a few times and then did a kill -3 on the PID to force a thread dump.

With Netty set as server provider: https://gist.github.com/KoenDG/b8f44187e148d56804f4f9591c9c88c2

With nothing set as server provider: https://gist.github.com/KoenDG/015b209a04a3afb9828efed22b35390d

With Akka set as server provider: https://gist.github.com/KoenDG/4f7fe24c8f14e55889bdf8adaa3b2b6f

The first 2 look very similar, with the Akka one having some thread names not found in the first two. If someone more knowledgeable in these matters could have a look to verify all is ok, that would be appreciated.

Only real thing of note is that the once started with Netty print this every time a route is called:

[warn] p.c.s.NettyServer - Ignoring unknown Netty channel option: TCP_FASTOPEN
[warn] p.c.s.NettyServer - Valid values can be found at http://netty.io/4.0/api/io/netty/channel/ChannelOption.html and http://netty.io/4.0/api/io/netty/channel/epoll/EpollChannelOption.html```

(Rich Dougherty) #7

I think the netty-event-loop threads show that Netty is running. So it’s running for the Netty server provider, but not the Akka HTTP one, which looks correct so long as you’re supplying -Dplay.server.provider on the command line. If you don’t supply that value then it defaults to the value of play.server.provider in a reference.conf file on the command line.

These values are set inside reference.conf files in the Play akka-http and netty JARs, so you should only have one of these on the classpath, otherwise the server that is run will depend on which JAR is first on the command line. Can you list out the JARs to see if there is more than one server JAR on the classpath?

Looks like TCP_FASTOPEN is an option if you use a native transport.