Hello, We are using Lagom 1.6.1 with Cassandra as our event store. When testing our application in performance env where we are writing over 500k events per hour, we observed some slowness after initial couple of hours and finally in the logs we are seeing our persistent entities are unable to load with AskTimeOutExceptions. The current timeout we set is 20 sec.
We are also purging our tag_views table after 1 hour and data in that table is compacted hourly as discussed here https://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
Do we need to use any compaction for messages table as well?
All the events we are inserting have a unique persistentId, which means we are not trying to load any existing entity from the DB.
ERROR akka.actor.OneForOneStrategy.$anonfun$applyOrElse$1(76) - Ask timed out on [Actor[akka://application/user/downloader#-1888536582]] after [20000 ms]. Message of type [com.pan.pcs.dlp.kafka.ObjectAndResourceMetadata]. A typical reason for AskTimeoutExceptionis that the recipient actor didn't send a reply.akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://application/user/downloader#-1888536582]] after [20000 ms]. Message of type [com.pan.pcs.dlp.kafka.ObjectAndResourceMetadata]. A typical reason forAskTimeoutException is that the recipient actor didn't send a reply.-->\tat akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:646)-->\tat akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:667)-->\tat akka.actor.Scheduler$$anon$7.run(Scheduler.scala:476)-->\tat akka.dispatch.internal.SameThreadExecutionContext$$anon$1.unbatchedExecute(SameThreadExecutionContext.scala:21)-->\tat akka.dispatch.BatchingExecutor.execute(BatchingExecutor.scala:125)-->\tat akka.dispatch.BatchingExecutor.execute$(BatchingExecutor.scala:116)-->\tat akka.dispatch.internal.SameThreadExecutionContext$$anon$1.execute(SameThreadExecutionContext.scala:20)-->\tat akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:355)-->\tat akka.actor.LightArrayRevolverScheduler$$anon$3.executeBucket$1(LightArrayRevolverScheduler.scala:306)-->\tat akka.actor.LightArrayRevolverScheduler$$anon$3.nextTick(LightArrayRevolverScheduler.scala:310)-->\tat akka.actor.LightArrayRevolverScheduler$$anon$3.run(LightArrayRevolverScheduler.scala:262)-->\tat java.base/java.lang.Thread.run(Unknown Source)