Actor Cluster Slowness when Large size messages produced by a ShardRegion proxy

We implemented Akka clustering with cluster sharding for a use case.

When we doing load testing for that, We created 1000 entity actors in a Node by cluster Sharding.(Cluster Node)
And we sends messages to that entity actors from a Proxy Node (Shard Region Proxy on other Node).

What we done


akka {
 remote {
  netty.tcp {
      hostname = "x.x.x.x"
      port = 255x

akka.cluster {
        remember-entities = on
        remember-entities-store = ddata
        distributed-data.durable.keys = []
  • Created a Dispatcher thread with 1000 threads and assigned that to Entity actors.(Which is on Cluster Node).
  • Created a java program which spawn 100 threads and each thread produce message to 10 actors sequentially one by one by the ShardRegion Proxy from Proxy node to Cluster Node.
  • For each message we wait for acknowledgement from the Entity Actor to the sender thread.Thereafter only next message will be produced.
  • So at a time 100 parallel messages can be fired.

When i produce 10KB messages with this 100 Parallel threads to 1000 Entity Actors we getting the acknowledgement from Entity actor pretty <40 ms
But when i sending 100KB messages like the same the acknowledgement making 150 to even 200ms delay for each messages.
I know huge message will take more time than small messages.
As i read some blogs and others questions similar like this. They are saying to increase

akka {
     remote {
         netty.tcp {
             # Sets the send buffer size of the Sockets,
             # set to 0b for platform default
             send-buffer-size = 2MiB
             # Sets the receive buffer size of the Sockets,
             # set to 0b for platform default
             receive-buffer-size = 2MiB

this configurations.
Even after increased this config from 200KB,2MB,10MB,20MB there is no performance gain.

I put some debug log on Endpoint Writer Actor and saw a strange thing, even i have a buffer size 2MB when huge no of messages send to shard Region the Buffer in Endpoint writer is increasing but it writing into the Association-handle one by one.I’m getting logger for each message write to Association Handle(Same Association handle Object Id on each write).
Then is it sequential???
Then how the send and receive buffer used in this cases.?

Some one said increasing Shard count will help.Even after increasing there is no performance gain.
Is that any miss configuration i done or any Configuration i missed?.

Cluster Node have 1000 Entity Actors which split into 3 Shards.
Proxy Node which have 100 parallel threads which produce messages to the Cluster Node.

This seems to be a duplicate post.

1 Like

I edited with some changes… But even i didn’t get any answers for both questions.

Without sharing code and a full configuration of an example that exhibits this behavior, I’m not sure the question is answerable (especially in view of it being against a fairly old Akka version.

1 Like

The classic remoting based on Netty is deprecated, I strongly recommend that you instead use the new remoting if building a new application with Akka.

1 Like

Thank you johanandren.
Which is supported on 2.5.23 ?? Or should i move to latest version i can’t able to move to scala 2.12 or 13 nowadays because so many dependency in my project on other services so which is the best version to use with scala 2.11 for artery?

Artery was added in Akka 2.5.0 but do note that a long time has passed since 2.5.32 which was the last 2.5 release. Bug and security fixes as well as performance improvements has only happened to the active 2.6 branch since then, and we are not actively maintaining 2.5 anymore. (Note that this will also soon change for for 2.6 after 2.6.20 except critical issues however Akka 2.7 will be a drop in replacement for 2.6).