Server Sent Event and Source : Connection get "Reset" after page refresh

streams

(Matthieu Totet) #1

Hello,

I’m trying to do some web application using the “Server Sent Event” feature of Akka HTTP.

I followed the docs here : https://doc.akka.io/docs/akka-http/current/sse-support.html
And it works well. The entrypoint response correctly , my javascript can subscribe and receive the messages and, important on my case, if I refresh the page, it will reconnect.

So I extended from this point for my application. Basically, the idea is grabing some tweets from Twitter4J, doing some text processing and send the results to the SSE endpoint.

So I created a Source like this to get the tweets and process it.

val(emojiSourceMat,emojiSource) = Source.queue[Status](100000, OverflowStrategy.dropNew)
    .map(s=>EmojiParser.extractEmojis(s.getText).toArray.distinct.toSeq.mkString(" "))
    .preMaterialize()

Then each time the program receive a tweet it offers it to the source :

emojiSourceMat.offer(status)

And I use the source also for the sse entry point

complete {
           Boot.emojiSource.map(x=> ServerSentEvent(x))
}

And it works, the webpage will receive the messages. If I launch the same webpage multiple time it will works.

But if I have only one page and I refresh it, the javascript returns an error that it cannot connect to the entrypoint. And if I got to the entrypoint directly from the browser, it will says that The connection has been reset.
I’m aware that I might not use correctly all the streams correctly but I can’t really see where the problem is coming from.


(Matthieu Totet) #2

Ok clearer information to have maybe a hint on the issue.

I’ve created a project here :

I based first part from this blog (+some adaptation as it was from old version of akka)
http://loicdescotte.github.io/posts/play-akka-streams-twitter/

Here I’m creating a ActorRef Source that will receive some event and put in a Sink Publisher

Based on the doc https://doc.akka.io/docs/akka-http/current/sse-support.html , I’m creating a Source from the Publisher that generate a SSE Stream

I’m consuming the data here on client side

This hacky extra line makes the application works as expected

If this line is commented/removed , I can connect a first time, open as many webpage in parallel without any issue. But as long as no more page are loading the data, the Source looks stopped and it’s generating this logs

INFO] [10/23/2018 08:29:09.858] [wall-emoji-akka.actor.default-dispatcher-2] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.860] [wall-emoji-akka.actor.default-dispatcher-14] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [2] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.863] [wall-emoji-akka.actor.default-dispatcher-14] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [3] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.863] [wall-emoji-akka.actor.default-dispatcher-14] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [4] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.871] [wall-emoji-akka.actor.default-dispatcher-2] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [5] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.877] [wall-emoji-akka.actor.default-dispatcher-2] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [6] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.891] [wall-emoji-akka.actor.default-dispatcher-2] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [7] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.891] [wall-emoji-akka.actor.default-dispatcher-14] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [8] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.892] [wall-emoji-akka.actor.default-dispatcher-14] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [9] dead letters encountered. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[INFO] [10/23/2018 08:29:09.892] [wall-emoji-akka.actor.default-dispatcher-14] [akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource] Message [twitter4j.StatusJSONImpl] without sender to Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877] was not delivered. [10] dead letters encountered, no more dead letters will be logged. If this is not an expected behavior, then [Actor[akka://wall-emoji/system/StreamSupervisor-0/flow-0-2-actorRefSource#1389184877]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

As far as I understand, its closing event the ActorRef Source if no more client as consuming the last SSE Source.
I can understand that the Source generated per connection is closed, but why going up all along the graph to close everything ?
Is there a better way to avoid this ActorRef to be closed ? Or what is wrong with the code to behave like this ?