Recommended way to model a long-living worker?


what would be a proper way to model a long-living worker with akka? I have two persistent actors polling a database, for inserts and updates, each with a hard-coded unique persistence id. The intention is to have exactly one of both at any time. The persistent state represents the polling offset.
The actors are supervised with restartWithBackoff strategy. Also the actors are reacting to the PreRestart signal by closing the database connection.

My problem is that if an actor crashes, the new instance is not created with following warning:

{“level”:“WARN”,“logger”:“akka.persistence.cassandra.journal.TagWriters”,“message”:“Persistent actor starting for pid [UpdatesPollingCommand|UpdatesPollingCommand]. Old ref hasn’t terminated yet: [Actor[akka://Shipments/system/sharding/UpdatesPollingCommand/0/UpdatesPollingCommand#-1167587395]]. Persistent Actors with the same PersistenceId should not run concurrently”}

I’ve created a gist representing the important parts of my design here: actors.scala · GitHub

I’d be very thankful for any suggestions if something is wrong with my design/approach or with the code…

Could potentially be a bug in the Cassandra journal tag writer wrt persistent actor recovery. I’ve created an issue in the tracker: Tag writer warning when event sourced actor restarts · Issue #861 · akka/akka-persistence-cassandra · GitHub

BTW, unrelated, if you only start sharding on some role nodes you need to configure sharding with that role as well or it won’t work.

Thank you for your feedback!
Yes, I have few nodes configured with different roles and a dedicated single “intake”-role node so that I can be sure my workers aren’t moved across nodes.

The new instance will still be created, that warning doesn’t stop the restart happening. We should check if it is the same actor ref to detect if it is restarting