Initialize an entity (fixed service with an id) on startup

How could I pre-start an entity at cluster startup? I have found a way to do so but I think it is not the right way to do. It consists of sending a StartEntity(entityId) message to the shard region on every node. Suppose I have 1000 services to initialize. It seems very unperformant (explosion of messages in the cluster since every node tries to initialize the remote entity)!


        val shardRegion: ActorRef[ShardingEnvelope[Command]] =
        sharding.init(Entity(HelloServiceEntity)(createBehavior = ctx => HelloWorldService()))

        Seq("S0", "S1").foreach { id =>
          shardRegion ! StartEntity(id)
        }

Is there any efficient way to achieve what I want? I could not find an official post or documentation about it.

I had an idea ! I could use a cluster Singleton whose job would be to initialize entities. That’s the most efficient way I came up with without getting into internals and having to propose a pull request :joy:

Hey Lucas,

I don’t see anything wrong with starting those actors that you require upon start-up by messaging them, it’s pretty much what Sharded Daemon Process does. You and the sharded daemon process documentation agree that this ain’t performant, and should be used with care.

What is the use case where you have to start thousands of actors upfront, keeping them alive all the time?

1 Like

I am implementing kind of a distributed ACID event store and I need some set of actors to initialize upfront to consume commands from a Kafka topic…

What do you think about my second idea to use a cluster singleton to initialize the entities ? (my own reply) ?

Hey Lucas,

I would still pick Sharded Daemon Process over singleton, since such consumers are its canonical use case. It does actually work rather well - and considering how quick the Kafka consumers are, you won’t require too many of them. I’ve used an “internal” backport of Sharded Daemon Process, prior to its introduction to the core, for consuming large volumes of messages from Kafka before. The “keep alive” messages of it were never an issue. Rest assured, you’ll be consuming the messages quicker than you can process and write them to their new store :slight_smile:

You might want to have a look at External shard allocation too, it sounds like it might be a good match to your case.

Borislav

1 Like