kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Kafka managed by zookeeper
Date Fri, 06 Jan 2012 18:10:09 GMT
>> is this an issue which is not implemented in Kafka or does zookeeper doesn't offer
this information?

As far as I know, this is how the zookeeper client works today. It has
to know the list of hosts that constitute a zookeeper cluster (though
the connection url), it caches the IPs locally, and uses plain round
robin to pick the next server to connect to.


On Fri, Jan 6, 2012 at 2:16 AM, Pierre-Yves Ritschard <pyr@spootnik.org> wrote:
> Well, I am running zookeeper on EC2 and rely on configuration
> management to update configuration files when topology changes. Chef
> and Pallet in particular are very good at addressing that type of
> problem.
> As for haproxy shifting the problem, you can rely on elastic ips and
> two load-balancing instances. If you want to keep the moving parts and
> types of nodes down and don't already run a load-balancing service,
> the CM approach is your best bet
> On Fri, Jan 6, 2012 at 11:04 AM, Tobias Schulze-Heinrichs
> <t.schulze@xplosion.de> wrote:
>> Hmm, that shift the problem just to another resource. If the haproxy instance crashes
I'll need to reconfigure the producer and consumers. I guess that problem needs to be addressed
to the experts of cloud computing.
>> Beside that I read that the zookeeper guys just working on a way to dynamically add
and/or remove zookeeper servers to an existing ensemble.
>> Best regards
>> Tobias
>> Am 06.01.2012 um 09:18 schrieb Pierre-Yves Ritschard:
>>> or you could load-balance access to your ensemble using haproxy or elb
>>> On Fri, Jan 6, 2012 at 9:06 AM, Tobias Schulze-Heinrichs
>>> <t.schulze@xplosion.de> wrote:
>>>> Hi Neha,
>>>> is this an issue which is not implemented in Kafka or does zookeeper doesn't
offer this information? To describe this problem a little bit more detailed - we want to set
up a service at the cloud (amazon ec2). Therefor it might occur that one instance crashes
and came up with a new IP. In that scenario I have to reconfigure and restart all Zookeeper
and all Kafka server, right?
>>>> Best regards
>>>> Tobias
>>>> Am 05.01.2012 um 21:39 schrieb Neha Narkhede:
>>>>> Tobias,
>>>>> Please configure Kafka with the list of all 3 zookeeper servers. The
>>>>> way the zookeeper client works is it just picks one zookeeper server
>>>>> from the list of all zookeeper servers in the connection url. If you
>>>>> miss specifying one server in your connection url, Kafka will never
>>>>> connect to the 3rd server, causing imbalance on the zookeeper cluster.
>>>>> Thanks,
>>>>> Neha
>>>>> On Thu, Jan 5, 2012 at 12:34 PM, Tobias Schulze-Heinrichs
>>>>> <t.schulze@xplosion.de> wrote:
>>>>>> Hello all,
>>>>>> I have an ensemble of 3 ZK Server. Do I need to configure all 3 zookeeper
servers to my Kafka server or is there a possibility that it can get the address of 2 ZK out
of one which is given at my Kafka configuration?
>>>>>> Best regards
>>>>>> Tobias

View raw message