Different number-of-shards per entity type

Akka
1

I’m using Akka sharding to store two different types.

  1. One of the types allocates a relatively small data set and small size (but still sharding is necessary).
  2. The other one allocates a lot of data and a huge size.

The problem that I’m going to have is that using the name number-of-shards for both, I will have a lot of really small size entities for the first case.

Is there a way to set the number-of-shards parameter per entity type? If it is not possible, what is the right way to approach it? Maybe writing a custom ShardingMessageExtractor

Thanks

2

You can pass in ClusterShardingSettings when initializing the entity type and override the config like:

ClusterShardingSettings.fromConfig(
  ConfigFactory.parseString("number-of-shards = 100).withFallback(
    system.settings.config.getConfig("akka.cluster.sharding")))

We should add a withNumberOfShards to ClusterShardingSettings to avoid the config dance.

One thing that you loose when defining a different number-of-shards programmatically is the compatibility check that is performed when nodes are joining the cluster. The setting must be the same on all nodes.

3

That is exactly what I was looking for.

Checking the source code, there is a comment:

  // no withNumberOfShards because it should be defined in configuration to be able to verify same
  // value on all nodes with `JoinConfigCompatChecker`

I’m 100% agree with your point about the withNumberOfShards.
I understand that there is only one ClusterSharding per system, so I think that my case is not so weird.

Maybe the akka.cluster.sharding configuration, or part of that, should be per Entity type. Or a default one and an option to overwrite values per type.