Access Cassandra on Kubernetes


(Zlatko) #1

Hi,

I’m trying to run online-auction-scala example on kubernates. Kubernates is installed as standalone on Linux box (so, it is not minikube).

kubectl get service:

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
bidding ClusterIP 10.110.88.131 10000/TCP,10001/TCP,10002/TCP 27h
cassandra ClusterIP 10.97.53.213 9042/TCP 19h
item ClusterIP 10.109.50.248 10000/TCP,10001/TCP,10002/TCP 27h
kafka ClusterIP 10.109.125.211 9092/TCP 28h
kafka-headless ClusterIP None 9092/TCP 28h
kafka-zookeeper ClusterIP 10.108.234.104 2181/TCP 28h
kafka-zookeeper-headless ClusterIP None 2181/TCP,3888/TCP,2888/TCP 28h
kubernetes ClusterIP 10.96.0.1 443/TCP 28h

kubectl exec -it busybox – nslookup cassandra.default.svc.cluster.local

Server: 10.96.0.10

Address: 10.96.0.10:53

Name: cassandra.default.svc.cluster.local

Address: 10.97.53.213

*** Can’t find cassandra.default.svc.cluster.local: No answer

A command to deploy bidding-imp:

rp generate-kubernetes-resources localhost:5000/bsi/biddingimpl:1.0.0-SNAPSHOT --generate-all --registry-disable-https --registry-disable-tls-validation
–pod-controller-replicas 2
–env JAVA_OPTS="-Dplay.crypto.secret=youmustchangeme4"
–env CASSANDRA_SERVICE_NAME=“cassandra.default.svc.cluster.local” | kubectl apply -f -

Then I find these errors in logs:

com.example.auction.bidding.impl.AuctionScheduler [] - Error running finish bidding query

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.97.53.213:0 (com.datastax.driver.core.exceptions.TransportException: [/10.97.53.213:0] Cannot connect))

Did I configure CASSANDRA_SERVICE_NAME well?
Does lagom trying to access Cassandra on port 0 ?

Zlaja


(Tim Moore) #2

It’s expecting you to configure with a full SRV domain name, which would include the correct port in the response.

This would look something like _cql._tcp.cassandra.default.svc.cluster.local, but the port name (_cql in this example) might be different depending on how you’ve configured your Cassandra Kubernetes Service.


(Eric Zuo) #3

I spend a few hours on this (deploy Lagom online auction with Cassandra on Kubernetes) and here is my rp command, hope it helps:

#Please note, cassandra is the service name in the Kubernetes namespace called taylor

export service_cassandra=_cql._tcp.cassandra.taylor.svc.cluster.local

#Deploy bidding-impl

rp generate-kubernetes-resources some.domain.of.image.host/biddingimpl:1.0.0
–generate-pod-controllers --generate-services
–env JAVA_OPTS="-Dplay.http.secret.key=$secret_bidding -Dplay.filters.hosts.allowed.0=$allowed_host"
–pod-controller-replicas 2
–registry-use-local
–transform-pod-controllers ‘.spec.template.spec.imagePullSecrets += [{name:“scalefunction-registry-secrets”}]’
–external-service “cas_native=$service_cassandra”
–external-service “kafka_native=$service_kafka” | kubectl apply -f -