Persistence failure when replaying events

akka-cluster

(michael) #1

Greetings,

we’re developing an large akka-cluster system that uses akka-persistence-cassandra with journal events and generates a snapshot every 10 events. after several days of running, i started seeing frequent entries like this in the log. the last known sequence number is always 0, and none of the unexpected seqNr are divisible by 10. i haven’t noticed any cluster or cassandra failures. any thoughts on what may have gone wrong? thank you.

Persistence failure when replaying events for persistenceId [104693c0-a89c-407f-a941-f22fe22797c7]. Last known sequence number [0]
java.lang.IllegalStateException: Sequence number [1] still missing after [10.00 s], saw unexpected seqNr [73] for persistenceId [104693c0-a89c-407f-a941-f22fe22797c7].

(Patrik Nordwall) #2

Do you delete events? Can you reproduce it by starting 104693c0-a89c-407f-a941-f22fe22797c7 again? What does the messages table look like for 104693c0-a89c-407f-a941-f22fe22797c7 if you query it with cqlsh?