Akka persistence R2DBC pubsub event dropped

Akka Persistence / Event Sourcing
1

Hi, while running application with r2dbc I’m observing in the log the following messages (turned on the log trace mode for it):

Feb 8, 2024 @ 13:24:03.840	Dropping pubsub event for persistenceId [Account-TC|TEN001:TEST-ACCOUNT-2] seqNr [27885] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:03.840	Dropping pubsub event for persistenceId [Account-TC|TEN001:TEST-ACCOUNT-2] seqNr [27885] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:03.840	Dropping pubsub event for persistenceId [Account-TC|TEN001:TEST-ACCOUNT-2] seqNr [27885] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:03.840	Dropping pubsub event for persistenceId [Account-TC|TEN001:TEST-ACCOUNT-1] seqNr [27676] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:03.840	Dropping pubsub event for persistenceId [Account-TC|TEN001:TEST-ACCOUNT-1] seqNr [27676] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:03.840	Dropping pubsub event for persistenceId [Account-TC|TEN001:TEST-ACCOUNT-1] seqNr [27676] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:00.911	Dropping pubsub event for persistenceId [TransactionCore|TEN001] seqNr [2715307] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:00.911	Dropping pubsub event for persistenceId [TransactionCore|TEN001] seqNr [2715307] because too far ahead of backtracking.
Feb 8, 2024 @ 13:24:00.911	Dropping pubsub event for persistenceId [TransactionCore|TEN001] seqNr [2715307] because too far ahead of backtracking.

It happens when persistence actor is idle for some time and then the first command is coming to the actor which results in saving of an event. Later the saved event is not immediately processed by projection, but “pubsub” dropped and processed on the next events query from the projection. It adds a delay (I think this one: akka.persistence.r2dbc.query.refresh-interval) to the event processing. The next events are processed as expected until actor again is for some time in idle mode.

It makes a problem for an app if traffic can come in batches by adding a delay to start time of the events processing.

Did anyone observe similar behavior?

2

Thanks for reporting. I think we can detect that idle situation and accept the pubsub event. Could you create an issue in Issues · akka/akka-persistence-r2dbc · GitHub ?

3

Sure, created: Dropping pubsub event in the logs · Issue #520 · akka/akka-persistence-r2dbc · GitHub