CPU high when migrating akka version from 2.5.25 to 2.6

Akka
1

Hello.
I’ve been used Akka 2.5.25 with cluster sharding. and it was already using artery.
I had 10 old(2.5.25) node and created new(2.6) one node to test 2.6 migration.

It looks clustering was successful and
both 2.5.25 and 2.6 can communicate each other.
But CPU of new node goes high around 60%. unfortunately I cannot find any special log about that.

I also tested with 10 new node and 1 old node.
At that time, only old node’s CPU goes high around 60%.

As I check CPU usage of linux,
akka.remote.default-remote-dispatcher-## thread was using many CPU.

Is there someone who faced same issue?
Do I have to add some configuration for stable migration?

Thanks.

2

Any chance you can profile it and share what is using so much CPU?

3

I used htop to check CPU usage.
And I found that 2 thread akka.remote.default-remote-dispatcher were using high CPU.
I forgot to get thread dump at that time :frowning:
Now all node of my service are consist of akka 2.6. so I need to create old node manually if you want to get other data.

4

Is this with Artery tcp or aeron-udp?

Do you have any custom configuration?

No hints of what is going from logs when enabling DEBUG level?

5 
remote {
    artery {
        enabled = on
        transport = tcp
        canonical.port = 2552
    }
}

I used above configuration for both 2.5.25 and 2.6.
Let me get DEBUG level log if we need to dig this issue.
I just had curious it was normal behavior.

6

Hi @i.na,

can you use jstack <pid> on the command line to see what the threads with high CPU are doing?

Johannes