Does anyone have any example of Complex Event Processing done using Akka in a HA fashion? CEP is listed as one of the use cases for Akka on the Akka website, but I don’t see any example of this provided anywhere.
The module of Akka that is most suited for such things is Akka Streams,
have you had a look at the intro / guide in the docs?
https://doc.akka.io/docs/akka/current/stream/stream-quickstart.html
1 Like
I implemented a large application (IoT domain; unfortunately closed source) based nearly entirely on Akka Streams. It is perfectly doable.
Are the events processable if they are time based? Or in your case, is each event processed separately by an actor?
How does HA work on it? Do you start a cluster and if half of it goes down, there is no impact on processing?
Can you point me to a simplified example?
The events and responses (coming from sensors) have timestamps and enter the system via RabbitMQ. A clustered application subscribes to a shared message queue. In case of “important” events they are acknowledged not before the critical processing took place. A crucial topic was to shutdown application instances gracefully and in a timely manner. (CloudFoundry gives you only 10 seconds to react on shutdown signals.) When all inputs (sources) of the application are completed then the completion signals propagate to the outputs (sinks). Then we can be sure that no more processing is in-flight. Critical events that where consumed but not completely processed get nacked and redelivered by RabbitMQ to other instances.
Difficulties arise if there a feedback cycles. I cut such cycles by a flow-like custom component that in case of shutdown signals unbounded demand upstream and completion downstreams. All received data during shutdown is persisted and picked up by other application instances.