Circuit breaker and business validations

Hi,

Since Lagom doesn’t allow a service to return any of the Container(s), like Option, Try or Either, my main question is how to handle business validations without triggering the circuit breaker?

In our case, when a POST request is being fired, we need to do some validations (e.g. duplicate record etc. or other business validations). Now currently, if the validation failed, I was throwing a BadRequest, however, if the same (validation failing) request is fired multiple times, it trips the circuit breaker. So to avoid tripping it, I used the following code in my application.conf file (as mentioned in the documentation) -

lagom.circuit-breaker {

  # Default configuration that is used if a configuration section
  # with the circuit breaker identifier is not defined.
  default {
    # Possibility to disable a given circuit breaker.
    enabled = on

    # Number of failures before opening the circuit.
    max-failures = 10

    # Duration of time after which to consider a call a failure.
    call-timeout = 10s

    # Duration of time in open state after which to attempt to close
    # the circuit, by first entering the half-open state.
    reset-timeout = 15s

    # A whitelist of fqcn of Exceptions that the CircuitBreaker
    # should not consider failures. By default all exceptions are
    # considered failures.
    exception-whitelist = ["com.lightbend.lagom.scaladsl.api.transport.BadRequest", "com.lightbend.lagom.scaladsl.api.transport.NotFound"]
  }
}

Please note that I’ve included BadRequest in the whitelist. Even after doing this, if I was firing the bad request 10 times (max-failures), the circuit breaker was getting tripped.

So, I included this in my Service class -

.withCircuitBreaker(
      CircuitBreaker.identifiedBy("default")
    )

Still it doesn’t help.

Finally, I created a ValidationFailed class, like -

object CustomTransportErrorCode {
  val BadData: TransportErrorCode = TransportErrorCode(600, -1003, "Invalid/Bad Data")
}

final class ValidationFailed(errorCode: TransportErrorCode, exceptionMessage: ExceptionMessage, cause: Throwable) extends TransportException(errorCode, exceptionMessage, cause) {
  def this(errorCode: TransportErrorCode, exceptionMessage: ExceptionMessage) = this(errorCode, exceptionMessage, null)
}

object ValidationFailed {
  val ErrorCode = CustomTransportErrorCode.BadData

  def apply(message: String) = new ValidationFailed(
    ErrorCode,
    new ExceptionMessage(classOf[ValidationFailed].getSimpleName, message), null
  )

  def apply(cause: Throwable) = new ValidationFailed(
    ErrorCode,
    new ExceptionMessage(classOf[ValidationFailed].getSimpleName, cause.getMessage), cause
  )
}

Please note, I’ve kept the Http code as 600 (because circuit breaker gets tripped by all HTTP 4xx and 5xx codes).

Even after doing all this, I am not able to avoid circuit breaker getting tripped due to business validation failures. Please let me know how to handle this?

@mmwaikar are you sure you are configuring CB on the right place (it has to be configured in the service that is using the api to initiate a call and not the one implementing it)? Can you try disabling CB to see what happens?

Thanks a lot Alan for your suggestion.

I removed all configuration from the implementing services and put it inside the web-gateway (from where all the other services were being called), and the whitelisting works fine for BadRequest. I didn’t need to put any code in the service definition or create any custom exception.

@aklikic How are application.conf merged across sub projects? Does it follow sbt project dependencies; e.g., if fooImpli depends on fooApi and core sub projects and not barApi or barImpl, would the application.conf be a merge of application.conf files found in fooImpl, fooApi and core?

Thanks, Damon

@dmrolfs application.conf is defined in impl not api. Impl should not depend on other impl so there is not merging to be done (there is no merging in any case).
Regarding CB config, config is always done in impl per used api (not api that impl implement but other services api).
I hope this answers your question.

Hi @aklikic,
I have some common configuration I am trying to localize (in a core sub-project) in order to avoid copy-pasting the same configuration. Normally, I’d use sbt-assembly plugin in order to merge application.conf definitions, but I’m unclear if that would conflict with the sbt-reactive-plugin, etc, used by Lagom.

Is there a recipe/guidance on how you can define (app domain or technical) configuration properties once and use across services jars?

Thanks in advance for you help!

Hi @dmrolfs,

It is always recommended to decouple services and avoid sharing.
Of course there are tradeoffs that can be used.
You could have a separate project (util project) that hosts shared configuration.
This project would be a service impl dependency and service impl application.conf would include general.conf.

Are you using or planning to use Multiple builds?

With multiple builds you create a separate util project with general configuration (published to repository) and include it as a dependancy in any service impl project.