We are currently building a CQRS micro-service for a Time & Attendance project. Our solution models Timecards to capture events like clock-in and clock-out. As a CQRS solution, we are also building read-side processors by hand using Akka Persistence and the akka-persistence-cassandra plugin, as we have not yet made the decision to move to Lagom.
The root of our problem
We want to be able to build each read-side Processor as a Persistent Actor in the cluster so that new Views play all the tagged events from the beginning of time.
If the schema for a given View evolves, we want to be able to truncate the Offset events for that view so that we can modify the View, and then replay all messages from the beginning of time, so that the modified View is up to date.
Akka Persistence and akka-persistence-cassandra, out of the box, creates a “messages” table (along with the supporting tag_view, etc tables). All the messages for all Persistent Actors are in that one table.
Question/Potential Solution:
Is there a way to create multiple instances of the akka-persistence-cassandra plugin and custom configure them, so that there are multiple “messages” tables with unique names?
We would want one messages table for the write-side events that would hold all the messages for all the Timecards. So, just one table to hold all the write-side events.
Then we would want one “messages” table per read-side Processor, so that the Offsets for each Processor have their own table. This way, each Processor can be replayed from the beginning to rebuild a given View independently of the others, simply by truncating its Offset event messages table.
It’s possible to use different plugin implementation in Akka Persistence and therefore it’s also possible to have two apart configurations, one for each kind of Persistent Actor and writing to different journals.
However, I would counter advice to use the journal as a replacement of a CRUD DB. Akka Persistence implements an even journal, saving offsets as events or snapshot is basically abusing it. It will give you a less than optimal solution.
Offset tracking is a simple CRUD operation, it makes no sense to have a journal for that. You should really consider an offset table and save the offset in the same transaction as your read-side model. If you make it a Persistent Actor like you described you will have at-most-once semantics and that’s totally not necessary.
It’s also much easier to drop an offset from a table than to drop the whole event history for a given view.
Have you also consider how this will behave in a cluster? Are you planning to shard the Persistent Actor you want to build to run the read-side? And if you shard them how you will initialise them and which events they will consume? You certainly don’t wont to have more than one consuming the same events.
Lagom solves that for you out-of-the-box and it supports sharding of tags so you can have many processors consuming each a subset of your events. The alternative to this is to have a singleton actor, but in that case, you create a bottleneck because you have only one consumer for all your events.
I advise you to reconsider this strategy and if you don’t want to use Lagom as you mentioned, at least have a look on how it is implemented there. It can be a good source of inspiration.