Cluster Design Question

Hi there,

I am newbie to akka. And I have a design question as below.

I am trying to design a payroll calculation framework as:
1- A main service node (role = ‘MainService’) which provides http APIs that receives payroll calculation request. The payroll calculation request will include:

  • Client ID/Name
  • Payroll Area ID
  • Employee List

2- When main service node receives the payroll calculation request, the request will be routed to backend calculator actors that distributed on backend nodes (role = ‘BackendCalculator’)

I might need to group backend calculator nodes based on different client ID. The backend nodes within one group will connect to the same database of that corresponding client (different group will connect to different databases). Also, this group of nodes will use a DSL codes that is scripted dedicated for that client. When payroll calculation request being routed to backend nodes, the router should dispatch the request based on different client ID. But how should I do this routing?

If I use normal cluster without sharding (only has roles), I will have to define a lot roles for different clients. And it is even worse that every time a new client joins the services, I will have to define new backend role like BACKEND_CLIENT_194, and change the routing code inside the main service node codes then re-complie. This might also requires the reboot of the whole main service instance which could cause unwanted downtime for the existing clients.

So basically the question is:
1- How can I dispatch calculation requests to group of nodes based on client ID?
2- For backend nodes in different groups, they should use different internal DSL codes and connect to different database.Is that possible?
3- When new client joins the service, how can I avoid re-build and reboot the main service node?

I have not learned cluster sharding yet, not sure if sharding will help in such scenario.

Thanks a lot for any sharing of thoughts.

Maybe you can use Group routers with different service keys for different client id. Then you can dynamically start backend actors and they become available for routing via the service key. See https://doc.akka.io/docs/akka/current/typed/routers.html#group-router

Hi Patrik,

Thanks so much for the idea. I am trying it out with your suggestion. Will update this post if it works perfect or in case that I got further issues :grinning:

@iLoveZod13 another, similar, option is to use work pulling https://doc.akka.io/docs/akka/current/typed/reliable-delivery.html#work-pulling

This is a new feature inAkka 2.6.4, which was why I didn’t mention it before.

Similar to the group router since you would have one service key per client id.