Using another execution context for server-side 3rd party API calls


(Julian Wilson) #1

Hi all,

I’m trying to do some server-side third-party API calling using play-ws on a job schedule in my play app. It’s using the quartz scheduler. It can be a lot of requests at once but I’m using monix’s AsyncSemaphore to limit the amount of calls to the third-party APIs so that is not an issue.

My question is would it be useful to use another execution context as per https://www.playframework.com/documentation/2.6.x/ThreadPools, or can I stick with the original one? If I were to use a new one, I would use the fork-join-executor as it’s really just IO.

Thanks!


(Adam Lane) #2

Not an answer to your question and I don’t know anything about AsyncSemaphore but I just wanted to make sure anyone reading this is aware of throttle in Akka Streams:
https://doc.akka.io/docs/akka/2.5.5/scala/stream/stream-quickstart.html#time-based-processing

I see an example of this in some of my old code just incase it helps people searching for best way to limit stuff:

val items = Vector() // of YourItemClass
val source: Source[YourItemClass, NotUsed] = Source(items)
val throttling = source
  .throttle(1, 1.second, 1, ThrottleMode.Shaping)
  .runForeach { yourItemClass =>
    // TODO: handle errors better in the case we want to continue streaming on error
    // I assume I added try because I found that error stopped iterating which was undesirable
    try {
      somethingWithYourItem(yourItemClass)
      // TODO: consider implications that this may just throttle creation of future rather than completion
    } catch {
      case e: Exception =>
        Logger.error("Something happened during iterating of throttle", e)
    }

  }(materializer)

throttling.map { _ =>
  Logger.info("throttling done")
}

(Julian Wilson) #3

Thanks for the response Adam. My question is more related to thread pools in Play but this is a nice snippet that I will use!