I have a set of micro-services such that I have one Orchestrator actor and one or more Worker actors. The orchestrator monitors Kafka topics and hands them out to workers to be consumers using alpakka. Each of these micro-services are their own Kubernetes pods.
My goal is to leverage something like the consumer lag metric, mentioned in the article below, to scale worker replicas as needed.
I have read elsewhere, though, that this metric may not be an accurate representation of things as alpakka clients may have cached messages that won’t show up in the lag metric. I have two questions from this:
- Is this a real concern and, if so, what’s a better metric to use?
- How do I get at these metrics via alpakka since I’m not using Akka Streams as the article documents? I don’t need to expose these such that K8s autoscales since I have an Orchestrator which could do the scaling so any, realtime access to one or more relevant metric is fine.