No configuration setting found for key 'decode-max-size'

Hi,

com.typesafe.config.ConfigException$Missing: No configuration setting found for key ‘decode-max-size’

I am getting this exception during the container boot-up. The only thing I changed since yesterday is that I upgraded the Lagom version from 1.4.8 to 1.4.9, and also upgraded the RP plug-in from 1.4.0 to 1.5.0.
Is there any extra configuration that needs to be defined in the application.conf because of the version upgrade?

Thanks,

This might indicate mis-matched library versions.

Can you provide more of the stack trace @lejoow?

Hi TimMoore,
I am facing a similar issue when trying to deploy the “online-auction-scala” example to minikube using the steps provided in the Kubernetes.md file (Operations Workflow)

Ref: https://github.com/lagom/online-auction-scala/blob/master/KUBERNETES.md

Please find the entire Stack Trace below:
(Note: I have only deployed bidding-impl service and the following belongs to the same):

Oops, cannot start the server.
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'decode-max-size'
	at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:156)
	at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:174)
	at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:188)
	at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:193)
	at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:250)
	at akka.http.impl.util.EnhancedConfig$.getPossiblyInfiniteBytes$extension(EnhancedConfig.scala:45)
	at akka.http.impl.settings.RoutingSettingsImpl$.fromSubConfig(RoutingSettingsImpl.scala:34)
	at akka.http.impl.settings.RoutingSettingsImpl$.fromSubConfig(RoutingSettingsImpl.scala:26)
	at akka.http.impl.util.SettingsCompanion.apply(SettingsCompanion.scala:51)
	at akka.http.scaladsl.settings.RoutingSettings$.apply(RoutingSettings.scala:46)
	at akka.http.scaladsl.settings.RoutingSettings$.apply(RoutingSettings.scala:45)
	at akka.http.scaladsl.settings.SettingsCompanion.apply(SettingsCompanion.scala:19)
	at akka.http.scaladsl.settings.SettingsCompanion.apply$(SettingsCompanion.scala:19)
	at akka.http.scaladsl.settings.RoutingSettings$.apply(RoutingSettings.scala:45)
	at akka.http.scaladsl.settings.SettingsCompanion.default(SettingsCompanion.scala:20)
	at akka.http.scaladsl.settings.SettingsCompanion.default$(SettingsCompanion.scala:20)
	at akka.http.scaladsl.settings.RoutingSettings$.default(RoutingSettings.scala:45)
	at akka.management.AkkaManagement.start(AkkaManagement.scala:112)
	at com.lightbend.rp.akkamanagement.AkkaManagementAutostart.<init>(AkkaManagementAutostart.scala:28)
	at com.lightbend.rp.akkamanagement.AkkaManagementAutostart$.createExtension(AkkaManagementAutostart.scala:36)
	at com.lightbend.rp.akkamanagement.AkkaManagementAutostart$.createExtension(AkkaManagementAutostart.scala:33)
	at akka.actor.ActorSystemImpl.registerExtension(ActorSystem.scala:956)
	at akka.actor.ActorSystemImpl.$anonfun$loadExtensions$1(ActorSystem.scala:989)
	at scala.collection.Iterator.foreach(Iterator.scala:929)
	at scala.collection.Iterator.foreach$(Iterator.scala:929)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
	at scala.collection.IterableLike.foreach(IterableLike.scala:71)
	at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
	at akka.actor.ActorSystemImpl.loadExtensions$1(ActorSystem.scala:987)
	at akka.actor.ActorSystemImpl.loadExtensions(ActorSystem.scala:1001)
	at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:876)
	at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:864)
	at akka.actor.ActorSystemImpl._start(ActorSystem.scala:864)
	at akka.actor.ActorSystemImpl.start(ActorSystem.scala:885)
	at akka.actor.ActorSystem$.apply(ActorSystem.scala:246)
	at play.api.libs.concurrent.ActorSystemProvider$.start(Akka.scala:187)
	at play.api.libs.concurrent.ActorSystemProvider$.start(Akka.scala:149)
	at com.lightbend.lagom.scaladsl.server.ActorSystemProvider$.start(LagomApplicationLoader.scala:264)
	at com.lightbend.lagom.scaladsl.server.LagomApplication.actorSystem$lzycompute(LagomApplicationLoader.scala:238)
	at com.lightbend.lagom.scaladsl.server.LagomApplication.actorSystem(LagomApplicationLoader.scala:237)
	at play.api.BuiltInComponents.tempFileReaper(Application.scala:334)
	at play.api.BuiltInComponents.tempFileReaper$(Application.scala:334)
	at play.api.BuiltInComponentsFromContext.tempFileReaper$lzycompute(ApplicationLoader.scala:122)
	at play.api.BuiltInComponentsFromContext.tempFileReaper(ApplicationLoader.scala:122)
	at play.api.BuiltInComponents.tempFileCreator(Application.scala:335)
	at play.api.BuiltInComponents.tempFileCreator$(Application.scala:335)
	at play.api.BuiltInComponentsFromContext.tempFileCreator$lzycompute(ApplicationLoader.scala:122)
	at play.api.BuiltInComponentsFromContext.tempFileCreator(ApplicationLoader.scala:122)
	at play.api.BuiltInComponents.playBodyParsers(Application.scala:280)
	at play.api.BuiltInComponents.playBodyParsers$(Application.scala:279)
	at play.api.BuiltInComponentsFromContext.playBodyParsers$lzycompute(ApplicationLoader.scala:122)
	at play.api.BuiltInComponentsFromContext.playBodyParsers(ApplicationLoader.scala:122)
	at com.lightbend.lagom.scaladsl.server.LagomServerComponents.lagomServerBuilder(LagomServer.scala:85)
	at com.lightbend.lagom.scaladsl.server.LagomServerComponents.lagomServerBuilder$(LagomServer.scala:85)
	at com.lightbend.lagom.scaladsl.server.LagomApplication.lagomServerBuilder$lzycompute(LagomApplicationLoader.scala:221)
	at com.lightbend.lagom.scaladsl.server.LagomApplication.lagomServerBuilder(LagomApplicationLoader.scala:221)
	at com.example.auction.bidding.impl.BiddingApplication.lagomServer$lzycompute(BiddingServiceLoader.scala:19)
	at com.example.auction.bidding.impl.BiddingApplication.lagomServer(BiddingServiceLoader.scala:19)
	at com.lightbend.lagom.scaladsl.server.LagomApplication.<init>(LagomApplicationLoader.scala:243)
	at com.example.auction.bidding.impl.BiddingApplication.<init>(BiddingServiceLoader.scala:13)
	at com.example.auction.bidding.impl.BiddingApplicationLoader$$anon$2.<init>(BiddingServiceLoader.scala:31)
	at com.example.auction.bidding.impl.BiddingApplicationLoader.load(BiddingServiceLoader.scala:31)
	at com.example.auction.bidding.impl.BiddingApplicationLoader.load(BiddingServiceLoader.scala:29)
	at com.lightbend.lagom.scaladsl.server.LagomApplicationLoader.load(LagomApplicationLoader.scala:77)
	at play.core.server.ProdServerStart$.start(ProdServerStart.scala:51)
	at play.core.server.ProdServerStart$.main(ProdServerStart.scala:25)
	at play.core.server.ProdServerStart.main(ProdServerStart.scala)

@lejoow, are you only getting this when deploying in a container?

The same problem was reported on Gitter. Here is the stack trace.

Oops, cannot start the server.
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'decode-max-size'
    at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:156)
    at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:174)
    at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:188)
    at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:193)
    at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:250)
    at akka.http.impl.util.EnhancedConfig$.getPossiblyInfiniteBytes$extension(EnhancedConfig.scala:45)
    at akka.http.impl.settings.RoutingSettingsImpl$.fromSubConfig(RoutingSettingsImpl.scala:34)
       ...

It seems to come from Akka Http and is probably being added by reactive-lib. It will add two akka-http route, on the for the cluster formation and one for the health check. We are possibly mixing versions and causing that to happen.

Akka Http reference.conf file has is defined as:

akka.http.routing.decode-max-size = 8m

I didn’t check it further, but for some reason this is not being added.

@lejoow / @sourav0457, please add this property to your application.conf for the moment. That should work around the issue.

Cheers,

Renato

1 Like

Created an issue to further investigate this:

Thanks @octonato. I think that has solved the issue. However, I am having other errors. Please find the Stack trace below:

urceActorSystem=application] - Failed to unmarshal Kubernetes API response status [403 Forbidden]; check RBAC settings
spray.json.DeserializationException: Object is missing required member 'items'
	at spray.json.package$.deserializationError(package.scala:23)
	at spray.json.ProductFormats.fromField(ProductFormats.scala:60)
	at spray.json.ProductFormats.fromField$(ProductFormats.scala:50)
	at akka.discovery.kubernetes.JsonFormat$.fromField(JsonFormat.scala:11)
	at spray.json.ProductFormatsInstances$$anon$1.read(ProductFormatsInstances.scala:34)
	at spray.json.ProductFormatsInstances$$anon$1.read(ProductFormatsInstances.scala:26)
	at akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport.$anonfun$sprayJsonUnmarshaller$1(SprayJsonSupport.scala:30)
	at akka.http.scaladsl.util.FastFuture$.$anonfun$map$1(FastFuture.scala:23)
	at akka.http.scaladsl.util.FastFuture$.strictTransform$1(FastFuture.scala:41)
	at akka.http.scaladsl.util.FastFuture$.transformWith$extension1(FastFuture.scala:45)
	at akka.http.scaladsl.util.FastFuture$.map$extension(FastFuture.scala:23)
	at akka.http.scaladsl.unmarshalling.Unmarshaller.$anonfun$map$3(Unmarshaller.scala:26)
	at akka.http.scaladsl.unmarshalling.Unmarshaller.$anonfun$transform$3(Unmarshaller.scala:23)
	at akka.http.scaladsl.unmarshalling.Unmarshaller$$anon$1.apply(Unmarshaller.scala:58)
	at akka.http.scaladsl.unmarshalling.Unmarshal.to(Unmarshal.scala:25)
	at akka.discovery.kubernetes.KubernetesApiSimpleServiceDiscovery.$anonfun$lookup$5(KubernetesApiSimpleServiceDiscovery.scala:96)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:304)
	at akka.http.scaladsl.util.FastFuture$.$anonfun$transformWith$1(FastFuture.scala:37)
	at akka.http.scaladsl.util.FastFuture$.strictTransform$1(FastFuture.scala:41)
	at akka.http.scaladsl.util.FastFuture$.transformWith$extension1(FastFuture.scala:45)
	at akka.http.scaladsl.util.FastFuture$.transformWith$extension0(FastFuture.scala:37)
	at akka.http.scaladsl.util.FastFuture$FulfilledFuture.transformWith(FastFuture.scala:86)
	at scala.concurrent.Future.flatMap(Future.scala:303)
	at scala.concurrent.Future.flatMap$(Future.scala:303)
	at akka.http.scaladsl.util.FastFuture$FulfilledFuture.flatMap(FastFuture.scala:77)
	at akka.discovery.kubernetes.KubernetesApiSimpleServiceDiscovery.$anonfun$lookup$4(KubernetesApiSimpleServiceDiscovery.scala:91)
	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:304)
	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
	at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
	at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.util.NoSuchElementException: key not found: items
	at scala.collection.MapLike.default(MapLike.scala:232)
	at scala.collection.MapLike.default$(MapLike.scala:231)
	at scala.collection.AbstractMap.default(Map.scala:59)
	at scala.collection.MapLike.apply(MapLike.scala:141)
	at scala.collection.MapLike.apply$(MapLike.scala:140)
	at scala.collection.AbstractMap.apply(Map.scala:59)
	at spray.json.ProductFormats.fromField(ProductFormats.scala:57)

Upon further investigation, I found the following thread:

https://github.com/lagom/online-auction-scala/issues/78

According to this, a file similar to this, needs to be added to the deployment process:

https://github.com/lagom/lagom-recipes/blob/master/kubernetes-deployment/hello-kubernetes-java-mvn/deploy/rbac.yml

using the command

kubectl apply -f RBAC.yml

Is this assumption correct? If yes, then could you please add this file to the online-auction-scala repo as well?

Thanks

Adding that property to the application.conf indeed fixed the issue.

For what it’s worth, we worked around this issue by adding a direct dependency on akka-http 10.1.5 in our -impl project.

Though it appears that simply adding the missing setting would have worked!

For anyone who might stumble across this thread -
The Error:

spray.json.DeserializationException: Object is missing required member ‘items’

can be solved by adding the RBAC.yml file as I had mentioned earlier.

For the record, it’s preferable to update the dependencies to akka-http 10.1.5 instead of simply add the property as I suggested.

As a rule of thumb, we should align the version for a given artifact family.

reactive-lib is bringing in akka-http and akka-http-spray-json v10.013 while Lagom 1.5.0 is bringing in akka-http-core 10.1.5.

The best fix for this now is to bump explicitly the versions for akka-http and akka-http-spray-json.

1 Like

For what it’s worth: we’re seeing this issue also when upgrading from 1.4.8 to 1.4.9, not just when running in a production setting, but even running a Lagom TestServer.

Adding that config value provides a successful workaround.