Akka-Kafka Cluster multiple consumer groups

Akka Akka Streams & Alpakka
1

Hi There,

As per Alpakka-Kafka documentation multiple consumer groups can be used to read from same topic but can be routed to different Behaviors.

I can’t wrap my head how this can be implemented, as per example:

akka-samples/UserEventsKafkaProcessor.scala at 2.6 · akka/akka-samples · GitHub

The shard region is passed via parameter then how can we route to different behavior?

Any example for this?

Regards,

Syed

2

Just to elaborate some more, my requirement is to route same message from one Kafka topic to different behaviors of different types, is this possible with different consumer groups?

The message structure is following:

message MyMessage {
oneof value {
Type1 t1 = 1;
Type2 t2 = 2;
}
}

Thanks

3

Hi @sfali23 . The example you referenced is to demonstrate Akka Cluster Sharding External Shard Allocation Strategies with Alpakka Kafka. It’s used with Akka Cluster Sharding to efficiently route consumed messages to Akka shard regions by rebalancing them to the same node as the Alpakka Kafka consumer.

It’s unclear to me if you’re using Akka Cluster Sharding or if you’re looking for a different example where you have two instances of an Alpakka Kafka stream consuming from different consumer groups. If it’s the latter, then you need to materialize two separate Akka Streams that have different ConsumerSettings (a different consumer group.id). Then in each stream you can transform and route messages to whatever typed actors you wish.