Long running job with progress

Hello!

I am using Play Framework/Java for quite a while now, but more like a traditional web framework with blocking calls. Never had any problems with it, because it was fast enough. I am not using the “advanced features” like Async, Streams, Akka etc.

Now, I have a new task to implement, and I think I have to use Akka for it. Please advise.

So, I have to download ca. 1.500 JPGs from a web service, create thumbnails and store the original and the thumbnail in the file system. After each 10 downloaded images, I want to show the user some progress.

I did that before with Jetty and a classic servlet, hat a loop and after each 10 images I did a “response.writeln(i + " images loaded”); response.flush();" and so the user could see what is going on.

How can I do something like that with Play?

I found https://stackoverflow.com/questions/11398495/long-running-process-with-progress-bar-example-playframework-2
Should I start with that?
The code is 3 years old, I think some methods changed. Is there a newer example anywhere?

Is there a better approach?

PS: Additionally, how could I stop the download, if the user wants to stop for any reason like after downloading 200 images?

Thank you,
schube

Hi @schube,

I think this is a perfect use case for a websocket backed by an actor. Here’s what I think you could do:

  • a websocket where the client may send commands (start, pause, cancel) to interact with the job
  • a parent actor that interacts with the websocket.
  • the actor creates a child actor to request the image URLs that need downloading.
  • having the URLS, the parent actor creates a handful of child actors and distributes the load of processing each image (download, convert, upload, save, etc…)
  • as each child actor progresses, feedback is sent to the parent actor that may send status updates back via the websocket.

The UI would be responsible to maintain the websocket open (sending ping messages over the wire) otherwise when play detects a websocket unused for a certain period of time it’ll be closed.

Note that if you are deploying multiple instances of your play process, your image processing actor may become unreachable if the websocket is closed because the client may not be able to hit the same play process (depending on your HTTP load balancing rules).

Note that you may need to create the parent actor at bootup time so closing the websocket doesn’t kill the actor.

An advanced option, not necessary but useful, is to build a cluster with your play processes so each play process can interact with actors in the other ones. The other advantage of building a cluster is that a single websocket request could then create a parent actor and ditribute child worker actors across the whole cluster speeding the process.

Cheers,

Hello!

Thank you for your reply and for confirming my thoughts about the task.

I tried to implement it last week with Akka Typed, but for me personally the documentation was too unclear/confusing so after a couple of hours I gave up.

Maybe, if I have time, I will try again some day, when there are more examples of Akka Typed with Play online.

Thank you,
Bernhard