kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Kafka managed by zookeeper
Date Fri, 06 Jan 2012 18:10:09 GMT
>> is this an issue which is not implemented in Kafka or does zookeeper doesn't offer
this information?

As far as I know, this is how the zookeeper client works today. It has
to know the list of hosts that constitute a zookeeper cluster (though
the connection url), it caches the IPs locally, and uses plain round
robin to pick the next server to connect to.

Thanks,
Neha

On Fri, Jan 6, 2012 at 2:16 AM, Pierre-Yves Ritschard <pyr@spootnik.org> wrote:
> Well, I am running zookeeper on EC2 and rely on configuration
> management to update configuration files when topology changes. Chef
> and Pallet in particular are very good at addressing that type of
> problem.
> As for haproxy shifting the problem, you can rely on elastic ips and
> two load-balancing instances. If you want to keep the moving parts and
> types of nodes down and don't already run a load-balancing service,
> the CM approach is your best bet
>
> On Fri, Jan 6, 2012 at 11:04 AM, Tobias Schulze-Heinrichs
> <t.schulze@xplosion.de> wrote:
>> Hmm, that shift the problem just to another resource. If the haproxy instance crashes
I'll need to reconfigure the producer and consumers. I guess that problem needs to be addressed
to the experts of cloud computing.
>>
>> Beside that I read that the zookeeper guys just working on a way to dynamically add
and/or remove zookeeper servers to an existing ensemble.
>>
>> Best regards
>> Tobias
>>
>>
>> Am 06.01.2012 um 09:18 schrieb Pierre-Yves Ritschard:
>>
>>> or you could load-balance access to your ensemble using haproxy or elb
>>>
>>> On Fri, Jan 6, 2012 at 9:06 AM, Tobias Schulze-Heinrichs
>>> <t.schulze@xplosion.de> wrote:
>>>> Hi Neha,
>>>>
>>>> is this an issue which is not implemented in Kafka or does zookeeper doesn't
offer this information? To describe this problem a little bit more detailed - we want to set
up a service at the cloud (amazon ec2). Therefor it might occur that one instance crashes
and came up with a new IP. In that scenario I have to reconfigure and restart all Zookeeper
and all Kafka server, right?
>>>>
>>>> Best regards
>>>> Tobias
>>>>
>>>>
>>>> Am 05.01.2012 um 21:39 schrieb Neha Narkhede:
>>>>
>>>>> Tobias,
>>>>>
>>>>> Please configure Kafka with the list of all 3 zookeeper servers. The
>>>>> way the zookeeper client works is it just picks one zookeeper server
>>>>> from the list of all zookeeper servers in the connection url. If you
>>>>> miss specifying one server in your connection url, Kafka will never
>>>>> connect to the 3rd server, causing imbalance on the zookeeper cluster.
>>>>>
>>>>> Thanks,
>>>>> Neha
>>>>>
>>>>> On Thu, Jan 5, 2012 at 12:34 PM, Tobias Schulze-Heinrichs
>>>>> <t.schulze@xplosion.de> wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> I have an ensemble of 3 ZK Server. Do I need to configure all 3 zookeeper
servers to my Kafka server or is there a possibility that it can get the address of 2 ZK out
of one which is given at my Kafka configuration?
>>>>>>
>>>>>> Best regards
>>>>>> Tobias
>>>>
>>

Mime
View raw message