I’m looking at transitioning an existing codebase to Alpakka Kafka and one of the requirements is that a single Kafka message can result in the output of 100k+ of Kafka messages. It also requires at-least-once delivery semantics. This is currently done with Scala
views so that the producer messages are produced lazily and don’t consume vast amounts of memory. Looking at Alpakka, the
ProduceMessage.multi takes an immutable
Seq which is not consistent with
view. I could use a
Stream which would use lazy evaluation but would still have the problem with the memory footprint.
Is there any way to to solve this problem?