I have a question about the granularity of events for read side processor for inserting data into Cassandra.
I could, of course, put what I want to write into one event, and then in read side processor generate N prepared statements (N = A + B + C…) and write them.
I am more interested in another approach, where I’d split my original event into more smaller ones, where each would be translated to A, B, C… prepared statements.
Am I right to assume that I’d have multiple read-side consumers/writers in runtime in production, which could share the load of writing data into Cassandra? I don’t really need atomic/batch inserts, but I do need them to eventually end up there.
ReadSide processor processes events in sequence by tag.
One entity instance events uses one tag.
This mechanism is used to preserve event order.
So if events are from the same instance, you will not get parallelism more then 1 in either options.
If order is not important but high throughput/parallelism you could use Kafka but with different partitioning key strategy then entity id.
So, the semantics of a ReadSide processor are similar to those of Kafka Consumer and how consumer groups work. Events tagged with a certain value will only be processed on one ReadSide instance at a time, got it.
I’ll definitely think about changing the partitioning key, thanks!!