Akka Kubernetes Error

Hi,
I’m facing issue in akka cluster on kubernetes.

[Cluster(akka://vishnu-controller)] - Cluster Node [akka://vishnu-controller@127.0.0.1:2550] - Started up successfully
[2020-07-28 15:58:43,431] [INFO] [akka.cluster.Cluster] [vishnu-controller-akka.actor.default-dispatcher-5] [Cluster(akka://vishnu-controller)] - Cluster Node [akka://vishnu-controller@127.0.0.1:2550] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining
[2020-07-28 15:58:43,444] [INFO] [akka.cluster.sbr.SplitBrainResolver] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/cluster/core/daemon/downingProvider] - SBR started. Config: stableAfter: 20000 ms, strategy: LeaseMajority, selfUniqueAddress: UniqueAddress(akka://vishnu-controller@127.0.0.1:2550,2743082301543251438), selfDc: default
[2020-07-28 15:58:43,834] [INFO] [akka.cluster.sharding.typed.scaladsl.ClusterSharding] [vishnu-controller-akka.actor.default-dispatcher-5] [ClusterSharding(akka://vishnu-controller)] - Starting Shard Region Proxy [Dummy] (no actors will be hosted on this node) for role [Some(write-model)] and dataCenter [None] …
[2020-07-28 15:58:43,881] [INFO] [akka.cluster.sharding.ShardRegion] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller@127.0.0.1:2550/system/sharding/DummyProxy] - Dummy: Idle entities will be passivated after [2.000 min]
[2020-07-28 15:58:43,908] [INFO] [akka.management.internal.HealthChecksImpl] [vishnu-controller-akka.actor.default-dispatcher-5] [HealthChecksImpl(akka://vishnu-controller)] - Loading readiness checks [(cluster-membership,akka.management.cluster.scaladsl.ClusterMembershipCheck), (example-ready,com.colortokens.vishnu.server.healthcheck.ServerHealthCheck)]
[2020-07-28 15:58:43,908] [INFO] [akka.management.internal.HealthChecksImpl] [vishnu-controller-akka.actor.default-dispatcher-5] [HealthChecksImpl(akka://vishnu-controller)] - Loading liveness checks []
[2020-07-28 15:58:43,987] [INFO] [akka.management.scaladsl.AkkaManagement] [vishnu-controller-akka.actor.default-dispatcher-5] [AkkaManagement(akka://vishnu-controller)] - Binding Akka Management (HTTP) endpoint to: 172.17.0.7:8558
[2020-07-28 15:58:44,068] [INFO] [akka.management.scaladsl.AkkaManagement] [vishnu-controller-akka.actor.default-dispatcher-5] [AkkaManagement(akka://vishnu-controller)] - Including HTTP management routes for ClusterHttpManagementRouteProvider
[2020-07-28 15:58:44,130] [INFO] [akka.management.scaladsl.AkkaManagement] [vishnu-controller-akka.actor.default-dispatcher-5] [AkkaManagement(akka://vishnu-controller)] - Including HTTP management routes for ClusterBootstrap
[2020-07-28 15:58:44,135] [INFO] [akka.management.cluster.bootstrap.ClusterBootstrap] [vishnu-controller-akka.actor.default-dispatcher-5] [ClusterBootstrap(akka://vishnu-controller)] - Using self contact point address: http://172.17.0.7:8558
[2020-07-28 15:58:44,202] [INFO] [akka.management.scaladsl.AkkaManagement] [vishnu-controller-akka.actor.default-dispatcher-5] [AkkaManagement(akka://vishnu-controller)] - Including HTTP management routes for HealthCheckRoutes
[2020-07-28 15:58:44,601] [INFO] [akka.management.cluster.bootstrap.ClusterBootstrap] [vishnu-controller-akka.actor.default-dispatcher-3] [ClusterBootstrap(akka://vishnu-controller)] - Initiating bootstrap procedure using kubernetes-api method…
[2020-07-28 15:58:44,613] [INFO] [akka.management.scaladsl.AkkaManagement] [vishnu-controller-akka.actor.default-dispatcher-5] [AkkaManagement(akka://vishnu-controller)] - Bound Akka Management (HTTP) endpoint to: 172.17.0.7:8558
[2020-07-28 15:58:44,614] [INFO] [akka.management.cluster.bootstrap.ClusterBootstrap] [vishnu-controller-akka.actor.default-dispatcher-5] [ClusterBootstrap(akka://vishnu-controller)] - Bootstrap using akka.discovery method: kubernetes-api
[2020-07-28 15:58:44,695] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/bootstrapCoordinator] - Locating service members. Using discovery [akka.discovery.kubernetes.KubernetesApiServiceDiscovery], join decider [akka.management.cluster.bootstrap.LowestAddressJoinDecider], scheme [http]
[2020-07-28 15:58:44,699] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:44,700] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-5] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:44,839] [INFO] [akka.actor.typed.ActorSystem] [scala-execution-context-global-25] [] - GRPC server started and bound to port : ServerBinding(/127.0.0.1:8080)
[2020-07-28 15:58:45,789] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:45,790] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-3] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:46,236] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-11] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:46,329] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:46,544] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-21] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-9.default.pod.cluster.local-8558] - Probing [http://172-17-0-9.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-9.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:46,648] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-21] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:46,692] [INFO] [akka.management.cluster.bootstrap.LowestAddressJoinDecider] [vishnu-controller-akka.actor.default-dispatcher-5] [LowestAddressJoinDecider(akka://vishnu-controller)] - Discovered [2] contact points, confirmed [0], which is less than the required [2], retrying
[2020-07-28 15:58:46,880] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-21] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:47,553] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-13] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:47,553] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-13] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:47,591] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:47,652] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-21] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-8.default.pod.cluster.local-8558] - Probing [http://172-17-0-8.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-8.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:47,702] [INFO] [akka.management.cluster.bootstrap.LowestAddressJoinDecider] [vishnu-controller-akka.actor.default-dispatcher-5] [LowestAddressJoinDecider(akka://vishnu-controller)] - Discovered [3] contact points, confirmed [1], which is less than the required [2], retrying
[2020-07-28 15:58:47,705] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-9.default.pod.cluster.local-8558] - Probing [http://172-17-0-9.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-9.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:47,983] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-21] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:47,984] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:48,701] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-12] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:48,701] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-12] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:48,710] [INFO] [akka.management.cluster.bootstrap.LowestAddressJoinDecider] [vishnu-controller-akka.actor.default-dispatcher-5] [LowestAddressJoinDecider(akka://vishnu-controller)] - Discovered [3] contact points, confirmed [1], which is less than the required [2], retrying
[2020-07-28 15:58:48,716] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-21] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:48,793] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-9.default.pod.cluster.local-8558] - Probing [http://172-17-0-9.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-9.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:48,841] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-8.default.pod.cluster.local-8558] - Probing [http://172-17-0-8.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-8.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:49,093] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-22] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:49,099] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:49,700] [INFO] [akka.management.cluster.bootstrap.LowestAddressJoinDecider] [vishnu-controller-akka.actor.default-dispatcher-5] [LowestAddressJoinDecider(akka://vishnu-controller)] - Discovered [3] contact points, confirmed [1], which is less than the required [2], retrying
[2020-07-28 15:58:49,881] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-11] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:49,882] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-11] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:49,894] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-21] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:50,025] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-11] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-8.default.pod.cluster.local-8558] - Probing [http://172-17-0-8.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-8.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:50,215] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-21] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:50,241] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-21] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:50,673] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:51,040] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:51,041] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-23] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:51,087] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:51,143] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-8.default.pod.cluster.local-8558] - Probing [http://172-17-0-8.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-8.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:51,147] [ERROR] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-8.default.pod.cluster.local-8558] - Overdue of probing-failure-timeout, stop probing, signaling that it’s failed
[2020-07-28 15:58:51,150] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller/system/bootstrapCoordinator] - Received signal that probing has failed, scheduling contact point probing again
[2020-07-28 15:58:51,395] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-5] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:51,399] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:51,889] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-3] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.9:47462: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:51,896] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:52,224] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:52,225] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-3] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:52,239] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:52,265] [WARN] [akka.management.cluster.bootstrap.internal.HttpContactPointBootstrap] [vishnu-controller-akka.actor.default-dispatcher-22] [akka://vishnu-controller@127.0.0.1:2550/system/bootstrapCoordinator/contactPointProbe-172-17-0-8.default.pod.cluster.local-8558] - Probing [http://172-17-0-8.default.pod.cluster.local:8558/bootstrap/seed-nodes] failed due to: Tcp command [Connect(172-17-0-8.default.pod.cluster.local:8558,None,List(),Some(10 seconds),true)] failed because of java.net.ConnectException: Connection refused
[2020-07-28 15:58:52,492] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-13] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:52,497] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-13] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:52,701] [INFO] [akka.management.cluster.bootstrap.LowestAddressJoinDecider] [vishnu-controller-akka.actor.default-dispatcher-11] [LowestAddressJoinDecider(akka://vishnu-controller)] - Exceeded stable margins but missing seed node information from some contact points [172-17-0-8.default.pod.cluster.local:0] (out of [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0])
[2020-07-28 15:58:53,081] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-11] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.9:47462: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:53,118] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-11] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:53,191] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-11] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.8:49952: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:53,424] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-13] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:53,425] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-13] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:53,458] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:53,684] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-13] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.7:39274: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:53,685] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-23] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:53,699] [INFO] [akka.management.cluster.bootstrap.LowestAddressJoinDecider] [vishnu-controller-akka.actor.default-dispatcher-11] [LowestAddressJoinDecider(akka://vishnu-controller)] - Exceeded stable margins but missing seed node information from some contact points [172-17-0-8.default.pod.cluster.local:0] (out of [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0])
[2020-07-28 15:58:54,018] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-12] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:54,275] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-3] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.9:47462: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:54,336] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/system/bootstrapCoordinator] - Contact point [akka://vishnu-controller@127.0.0.1:2550] returned [0] seed-nodes []
[2020-07-28 15:58:54,373] [INFO] [akka.management.cluster.bootstrap.contactpoint.HttpClusterBootstrapRoutes] [vishnu-controller-akka.actor.default-dispatcher-11] [HttpClusterBootstrapRoutes(akka://vishnu-controller)] - Bootstrap request from 172.17.0.8:49952: Contact Point returning 0 seed-nodes []
[2020-07-28 15:58:54,623] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-11] [akka://vishnu-controller/system/bootstrapCoordinator] - Looking up [Lookup(vishnu-controller,None,Some(tcp))]
[2020-07-28 15:58:54,623] [INFO] [akka.discovery.kubernetes.KubernetesApiServiceDiscovery] [vishnu-controller-akka.actor.default-dispatcher-11] [KubernetesApiServiceDiscovery(akka://vishnu-controller)] - Querying for pods with label selector: [app=vishnu-controller]. Namespace: [default]. Port: [None]
[2020-07-28 15:58:54,661] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/system/bootstrapCoordinator] - Located service members based on: [Lookup(vishnu-controller,None,Some(tcp))]: [ResolvedTarget(172-17-0-8.default.pod.cluster.local,None,Some(/172.17.0.8)), ResolvedTarget(172-17-0-9.default.pod.cluster.local,None,Some(/172.17.0.9)), ResolvedTarget(172-17-0-7.default.pod.cluster.local,None,Some(/172.17.0.7))], filtered to [172-17-0-9.default.pod.cluster.local:0, 172-17-0-8.default.pod.cluster.local:0, 172-17-0-7.default.pod.cluster.local:0]
[2020-07-28 15:58:54,702] [INFO] [akka.management.cluster.bootstrap.internal.BootstrapCoordinator] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/bootstrapCoordinator] - Initiating new cluster, self-joining [akka://vishnu-controller@127.0.0.1:2550]. Other nodes are expected to locate this cluster via continued contact-point probing.
[2020-07-28 15:58:54,710] [INFO] [akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef] [vishnu-controller-akka.actor.default-dispatcher-12] [akka://vishnu-controller/deadLetters] - Message [akka.management.cluster.bootstrap.internal.BootstrapCoordinator$Protocol$BootstrappingCompleted$] from Actor[akka://vishnu-controller/system/bootstrapCoordinator#-1611751782] to Actor[akka://vishnu-controller/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://vishnu-controller/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings ‘akka.log-dead-letters’ and ‘akka.log-dead-letters-during-shutdown’.
[2020-07-28 15:58:54,714] [INFO] [akka.cluster.Cluster] [vishnu-controller-akka.actor.default-dispatcher-12] [Cluster(akka://vishnu-controller)] - Cluster Node [akka://vishnu-controller@127.0.0.1:2550] - Node [akka://vishnu-controller@127.0.0.1:2550] is JOINING itself (with roles [dc-default]) and forming new cluster
[2020-07-28 15:58:54,714] [INFO] [akka.cluster.Cluster] [vishnu-controller-akka.actor.default-dispatcher-12] [Cluster(akka://vishnu-controller)] - Cluster Node [akka://vishnu-controller@127.0.0.1:2550] - is the new leader among reachable nodes (more leaders may exist)
[2020-07-28 15:58:54,729] [INFO] [akka.cluster.Cluster] [vishnu-controller-akka.actor.default-dispatcher-3] [Cluster(akka://vishnu-controller)] - Cluster Node [akka://vishnu-controller@127.0.0.1:2550] - Leader is moving node [akka://vishnu-controller@127.0.0.1:2550] to [Up]
[2020-07-28 15:58:54,736] [INFO] [com.colortokens.vishnu.server.infrastructure.Guardian$] [vishnu-controller-akka.actor.default-dispatcher-3] [akka://vishnu-controller/user/listener] - MemberEvent: MemberUp(Member(address = akka://vishnu-controller@127.0.0.1:2550, status = Up))
[2020-07-28 15:58:54,744] [INFO] [akka.cluster.sbr.SplitBrainResolver] [vishnu-controller-akka.actor.default-dispatcher-5] [akka://vishnu-controller/system/cluster/core/daemon/downingProvider] - This node is now the leader responsible for taking SBR decisions among the reachable nodes (more leaders may exist).

The cluter formed correctly so you can safely ignore the dead letter