kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre-Yves Ritschard <...@spootnik.org>
Subject Re: Kafka managed by zookeeper
Date Fri, 06 Jan 2012 10:16:41 GMT
Well, I am running zookeeper on EC2 and rely on configuration
management to update configuration files when topology changes. Chef
and Pallet in particular are very good at addressing that type of
As for haproxy shifting the problem, you can rely on elastic ips and
two load-balancing instances. If you want to keep the moving parts and
types of nodes down and don't already run a load-balancing service,
the CM approach is your best bet

On Fri, Jan 6, 2012 at 11:04 AM, Tobias Schulze-Heinrichs
<t.schulze@xplosion.de> wrote:
> Hmm, that shift the problem just to another resource. If the haproxy instance crashes
I'll need to reconfigure the producer and consumers. I guess that problem needs to be addressed
to the experts of cloud computing.
> Beside that I read that the zookeeper guys just working on a way to dynamically add and/or
remove zookeeper servers to an existing ensemble.
> Best regards
> Tobias
> Am 06.01.2012 um 09:18 schrieb Pierre-Yves Ritschard:
>> or you could load-balance access to your ensemble using haproxy or elb
>> On Fri, Jan 6, 2012 at 9:06 AM, Tobias Schulze-Heinrichs
>> <t.schulze@xplosion.de> wrote:
>>> Hi Neha,
>>> is this an issue which is not implemented in Kafka or does zookeeper doesn't
offer this information? To describe this problem a little bit more detailed - we want to set
up a service at the cloud (amazon ec2). Therefor it might occur that one instance crashes
and came up with a new IP. In that scenario I have to reconfigure and restart all Zookeeper
and all Kafka server, right?
>>> Best regards
>>> Tobias
>>> Am 05.01.2012 um 21:39 schrieb Neha Narkhede:
>>>> Tobias,
>>>> Please configure Kafka with the list of all 3 zookeeper servers. The
>>>> way the zookeeper client works is it just picks one zookeeper server
>>>> from the list of all zookeeper servers in the connection url. If you
>>>> miss specifying one server in your connection url, Kafka will never
>>>> connect to the 3rd server, causing imbalance on the zookeeper cluster.
>>>> Thanks,
>>>> Neha
>>>> On Thu, Jan 5, 2012 at 12:34 PM, Tobias Schulze-Heinrichs
>>>> <t.schulze@xplosion.de> wrote:
>>>>> Hello all,
>>>>> I have an ensemble of 3 ZK Server. Do I need to configure all 3 zookeeper
servers to my Kafka server or is there a possibility that it can get the address of 2 ZK out
of one which is given at my Kafka configuration?
>>>>> Best regards
>>>>> Tobias

View raw message