I am using Alpakka Kafka connector in the Source and using the flow to forward messages to an actor.
There is no configuration for restart in the Source in case of error, but I have observed that sometimes the messages from the Kafka topic are read from the beginning again. The “auto.offset.reset” property for the connector is set to “earliest”.
Hi @ullasdiwakar, welcome to the community =D.
In Akka has a supervision strategy to handle stream resumes, restart or drop in case off erros look at
https://doc.akka.io/docs/akka/2.5/general/supervision.html to know more about that and here https://doc.akka.io/docs/akka/current/stream/stream-error.html#supervision-strategies to get some nice examples.
The property “auto.offset.reset” is used when offset is not present or has been lost for any reason, so if you can assume data loss, use “latest”, besides that, check your consumer behavior. Look ate “at most once” and “at least one” semantics to use the right one to your application
Hi @ullasdiwakar,
Did you see the section about error handling in the Alpakka Kafka docs?
https://doc.akka.io/docs/alpakka-kafka/current/errorhandling.html
Alpakka Kafka consumers do not fail by default if the Kafka broker is unavailable. You may want to configure a connection checker. See connection-checker in reference.conf:
https://doc.akka.io/docs/alpakka-kafka/current/consumer.html#settings
Cheers,
Enno.