How do I configure CoordinatedShutdown timeout?

Play Framework
1

I’ve been using ApplicationLifecycle and I’m trying to replace it with CoordinatedShutdown. What I’m doing right now is moving my stophooks to CoordinatedShutdown.PhaseServiceStop(), and I’m seeing this warning:
Coordinated shutdown phase [service-stop] timed out after 5000 milliseconds

So is there a way around this? Can I configure the timeout somehow? Thanks!

2

You can see some examples of configuring the timeout in the Akka documentation:

https://doc.akka.io/docs/akka/2.5.3/scala/actors.html#coordinated-shutdown

For example:

akka.coordinated-shutdown.phases.service-stop.timeout = 10s
3

Thanks for the answer! But now I think I have 2 additional questions.

  1. Is it possible for me to register 2 stop hooks in service-stop phase with different timeouts per hook?
  2. According to the docs here, ApplicationLifecycle uses service-stop under the hood, so why am I not seeing the same timeout warning when I’m using ApplicationLifecycle?
4

Not per hook. All of the hooks within a phase are run concurrently, so you should set the timeout of the phase to the longest of the hooks you are running. If you need more flexibility than that, you can put hooks in different phases or create custom phases.

Note also that when a timeout triggers, it doesn’t halt the stop hook, it just moves on to the next phase. Eventually, the process will terminate after all phases complete or timeout.

Hard to say. I agree that you’d expect that if the behavior hasn’t changed. Are you actually expecting your hook to take longer than five seconds? Is it possible that the translation introduced a bug causing the returned Future not to complete? Can you post your code?

5

So I’ve actually been managing my own stop hooks and ordering, and I have a method private CompletionStage<Void> onStop(), and I was using lifecycle.addStopHook(this::onStop); to regster it with ApplicationLifecycle. And now with CoordinatedShutdown, this is what I’m doing:

	coordinatedShutdown.addTask(CoordinatedShutdown.PhaseServiceStop(), "stophook",
			() -> onStop()
			.handle((ignored, t) -> {
				if (t != null) {
					logger.error("", t);
				}
				return akka.Done.getInstance();
			}));
6

I don’t see an obvious reason why the Play stop hook would complete in time but the new coordinated shutdown task wouldn’t. I guess the interesting things happen within onStop, so it would be good to see that.

Has increasing the timeout helped?

7

Yes, increasing the timeout helped. Our onStop method has been taking over 5 seconds since the Play 2.5 days, and I’ve never seen warnings like that.

8

Play has only stopped its ApplicationLifecycle within an Akka Coordinated Shutdown phase since version 2.7.0. Prior to that, it would wait indefinitely for the stop hooks to complete. Could that explain the discrepancy? Did you try the original hook with Play 2.7.0 or later?

9

I did try the original hook with Play 2.7.x, and I’m fairly certain that I didn’t see the warning, but at least now I’ve successfully migrated to CoordinatedShutdown now.