Consume-Transform-(not always)Produce with Streams

java
kafka
streams
configuration

(João Gameiro) #1

Hi,

I was trying to implement a Kafka consumer that could produce a new message, if required, keeping it as transactional as possible.

Hence, at the moment, I have the implementation, with the retry mechanism, displayed on the page Transactions. Then, when I execute my map method and I don’t get a message to send, it (obviously) throws an exception.

I’ve already tried to use the splitWhen method (but, as far as I can see, this only serves to process messages in parallel, it is not possible to set a different behaviour for the predicate method) and there are insufficient examples that combine the divertTo method and the Alpakka implementation, so I was not able create a Graph where it’s sole purpose was to invoke the via method and send that message into Kafka

Could you please help me with this issue?

Without this feature I would not be capable of detecting duplicate messages without producing any sort of message into a topic, and thus creating a lot of garbage on my Kafka servers.

Thanks for the help

btw: I’m using Java on my implementation :slight_smile:


(João Gameiro) #2

Already solved this issue thanks to the discussion on link
:smile: