kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pritam Kadam <kpri...@thoughtworks.com>
Subject [Production Guideline] Is it good idea to provide limited number of brokers(3 brokers) in kafka client in large cluster
Date Wed, 17 Jan 2018 08:48:40 GMT
Hi,

I have a use case where my kafka cluster will have 1000 brokers and I am
writing kafka client.
In order to write client, i need to provide brokers list.

Question is, what are the recommended guidelines to provide brokers list in
client?

*Is there any proxy like service available in kafka which we can give to
client?*
 - that proxy will know all the brokers in cluster and connect client to
appropriate broker.
 - like in redis world, we have twemproxy (nutcracker)

*Is it recommended to provide any specific number of brokers in client, for
example provide list of 3 brokers even though cluster has 1000 nodes?*
 - what if provided brokers gets crashed?
 - what if provided brokers restarts and there location/ip changes?

Thanks,
Pritam.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message