drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaimes, Rafael - 0993 - MITLL" <Rafael.Jai...@ll.mit.edu>
Subject scaling drill in an openshift (K8s) cluster
Date Tue, 24 Mar 2020 13:33:54 GMT
I'm seeing a problem with scaling the number of pod instances in the
replication controller because they aren't reporting their hostnames
properly. This was a common problem that got fixed in scalable architectures
like ZooKeeper and Kafka (see reference at bottom I think this was related).


In Drill's case, ZooKeeper is able to see all of the drillbits, however, the
hostnames are only locally addressable within the cluster, so as soon as you
perform a query it fails since the client can't find the drillbit that it
got assigned, its hostname isn't externally addressable.


Kafka fixes this by allowing an override for advertised names. Has anyone
gotten Drill to scale in a K8s cluster?



View raw message