Akka sharding with consistent hashing instead of shard coordinator


#1

I have a question related to Akka.

Is there any way to use sharding like mechanism with consistent hashing?

I mean I would like to have one ActorRef per Akka node to which I can send a message that will be after forwarded to the appropriate receiver.
I know that there are consistent hashing group and pool, but they are not creating new actors base on incoming messages as Akka sharding does.
I want to use this approach as sometimes shard coordinator singleton is on Akka node under high load, and Akka cluster stops working.
With consistent hashing, there is no need for a coordinator as message are dispatched to actors/nodes base on hash derived from a message.

Is there any solution you could suggest?


(Johan Andrén) #2

This is exactly what sharding does. However you have to take into account that the topology of your cluster may change over time - nodes crashing, being removed - re-added etc. which is why sharding needs to have consistent state and rebalancing and cannot permanently tie a hash to a specific node.

Note that the interaction with the coordinator is just to figure out where a shard is, the general, happy case, will not require any coordination but the sending node will already know what node hosts a specific shard .

Looking into implementing your own ShardAllocationStrategy will likely provide some insights on this topic.