I’ve got a bit of an unusual circumstance and one of my services is talking directly to Kafka.
I’ve been trying to monitor the kafka topic in dev mode using the “kafka-console-consumer” provided by kafka, explicitly setting it to the only partition & topic I’ve allowed and telling it to play from offset “earliest”.
kafka-console-consumer.sh --topic=machine --bootstrap-server localhost:9092 --offset earliest --partition 0
For a brief window of about a minute, it successfully pulls down the messages, even several times, but after about a minute of running (after allowing the event-generating process to finish), re-running produces nothing, all of the old messages are gone.
Its my understanding that lagom’s dev-mode Kafka is supposed to be configured by default to retain messages for a while, the
log.retention.hours=168 line in the settings I thought meant it should last for a week.
I realize this may be more of a kafka problem than a lagom problem, but as I’m also worried about what it might do to other services relying on replaying those messages, and it’s lagom’s dev-mode configuration, I figured I’d ask here first, in case my basic assumptions were flawed.