Looking at the Kafka message broker implementation I see that when a service starts it begins publishing message for each of its topic to the broker, resuming from where it last left off. This gets all the messages into the broker independent of any particular client. It’s up to the broker implementation to manage each client’s consumption of messages (consumer groups, offsets, etc).
In the case of Kafka, if we don’t allow an infinite retention period then message will eventually be deleted. Is it the case that if a new service comes online after the retention period, it won’t be able to subscribe to a topic from the beginning because some messages will no longer be available? If so, are there any best practices on how to handle this case?