Distributed Publish Subscribe in Akka Typed Cluster

akka-typed

(Jonas Chapuis) #1

I’m running an akka-typed cluster (with sharding and persistence) and I need a publish/subscribe mechanism.

Is the untyped distributed publish/subscribe as documented in https://doc.akka.io/docs/akka/2.5/distributed-pub-sub.html still the recommended approach for these scenarios?

Or should I use the receptionist, for instance with a service key representing the topic and using Find to retrieve active listeners then sending them the message? If so, it would be nice to have pre-existing wrappers implementing this particular pub/sub scenario.

Also, does the receptionist work like ActorSelection and are registrations persisted in case of recovery or passivation? It would be important that recovered actors are still present in the listing, otherwise how can they receive new messages?

Thanks!


(Jonas Chapuis) #2

I have implemented this using the receptionist. See https://gist.github.com/jchapuis/c81c15adfacbe178ebb1b09e33b3aa67#file-servicekeybroadcaster-scala-L11

Let me know what you think, is this the proper approach?


(Johan Andrén) #3

There is no typed wrappers/APIs for pub-sub, I can’t remember if this was based on a decision that it is superflous with the receptionist in place or if it is just an oversight.

One thing that the distributed pub sub does which will differ from using the receptionist is that if there are multiple actors subscribing (registered for a service key) on the same node, the message is only sent to that node once and then passed to all subscribers.

To implement that kind of deduplication you would have to add some inbetween actor that registers once for the service key and keep track of the multiple local “subscribers”. Could be that this deserves some built in tool in Akka.

There’s some fishy things you do in that shared code: avoid attaching state to the service key and see it as just an identifier. Esp attaching something complex like the actor system to it will likely lead to problems with serialization. It also seems to use streams without no real benefit. It would be better to just ask the receptionist to get the listing back as a Future set (or subscribe to the key in an actor to get updates and keep a local set of known subscribers)

Edit: I created https://github.com/akka/akka/issues/26338 to track the idea of some built in tool


(Jonas Chapuis) #4

Thanks Johan for super quick and informative reply :+1:

Ok if I understand well, the receptionist-based approach as-is would generate potentially many messages across nodes. I would be indeed great to have a built-in mechanism then.

Yes I do like streams :wink: It could be interesting actually to have a built-in stream of subscriptions/unsubscriptions to a service key, what do you think? Here it’s just a single message stream with the listing.
I’m not sure I get your point about attaching state to the service key: is it related to the findCurrentListeners extension method I added? It was meant just as some helper to get the typed references of the current listeners.

We are also curious here about the persistence-related properties of the receptionist? will it keep references of passivated actors, or actors that are recovering?

Thanks again


(Johan Andrén) #5

I don’t think it makes sense to represent the subscriptions as a stream, what you get is an update with the new current set of subscribers, which are already in memory in the receptionist, so there isn’t really any gain with streams (nothing to backpressure etc)

Sorry about the service key, I skimmed it a bit too fast and read it as the extension class being the service key instance but I see now that is not how it is, so NVM that comment. :)

I did a small spike on a topic actor that you can find through the issue that may provide inspiration.