Hi all, we’re spinning up a microservice using Akka Persistence and Event Sourcing. We have a requirement for the service to be highly available across multiple regions and so we have an EKS cluster in each region and can deploy the service across clusters with global load balancing via Fastly.
However, my understanding of Akka Persistence and event sourced behaviours in particular is that actor state is held in memory while under the covers the events persisted to the journal are written into a database. As such, an actor in one region’s cluster would be unaware of changes made in another region as even with a global database, the database wouldn’t be in charge of co-ordinating updates across regions. To deal with that, you’d need a single Akka cluster spanning all of the regions so that events for a given persistence id are persisted in only one place which the whole system is aware of.
Given that the separate regions are separate kubernetes clusters, is there any guidance on how to have an Akka cluster span multiple k8s clusters? My understanding was that was discouraged though, so is there a recommended solution to this use case?