Lagom upgrading from Lagom 1.3.11 to 1.4.15, what to do with Kafka?

I have a few questions that I couldn’t find an answer for in the documentation nor somewhere online:

Some background information:

  1. We are not going to use the new ddata mode just yet, we’ll keep the default persistence mode
  2. Downtime is not much of an issue

Questions:

  1. Do you need to make sure that the Kafka bus is empty before doing an upgrade?
    1a. If so, what is the recommended strategy for this to happen?
    1b. If not, what will happen when doing a rolling upgrade when some nodes are still running 1.3 and some are already on 1.4? Will all the events still be able to be processed correctly?

Thank you in advance!

No, you don’t need to drain the Kafka topics before updating. The consumers will stop as each node shuts down, and resume on the new nodes as they start. The only thing required to ensure compatibility for Kafka consumers through a rolling upgrade is that your own deserialization and processing code doesn’t change in an incompatible way.

Note that there are other rolling upgrade considerations outlined in the migration guide:
https://www.lagomframework.com/documentation/1.4.x/java/Migration14.html#Upgrading-a-production-system

If downtime isn’t a concern for you, then a full shutdown and redeployment is a less risky and more predictable option compared with a rolling upgrade.

Thanks a lot Tim for your detailed response!