kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Carollo <ccaro...@gmail.com>
Subject Re: Kafka/ZK Cluster Example
Date Wed, 11 Jan 2012 22:35:46 GMT
I see.  But if I used that configuration and then did the mirroring you suggested would that
be enough, in your opinion, to be considered highly reliable?

Christian


On Jan 11, 2012, at 2:32 PM, Jun Rao wrote:

>> For example, can I have one ZK instance and one broker on one machine and
> that is enough to define a ZK cluster and a Kafka Cluster?
> 
> Yes, although you don't get the reliability of ZK now.
> 
> Jun
> 
> 
> On Wed, Jan 11, 2012 at 2:06 PM, Christian Carollo <ccarollo@gmail.com>wrote:
> 
>> Jun,
>> 
>> I don't think I ask my question the right way.
>> 
>> What I am trying to understand is what are the minimum constituent parts
>> of a kafka cluster?
>> 
>> Based on your last email, I am now wondering what are the minimum
>> constituent parts of a ZK cluster as well as a Kafka cluster?
>> 
>> For example, can I have one ZK instance and one broker on one machine and
>> that is enough to define a ZK cluster and a Kafka Cluster?
>> 
>> Thanks,
>> Christian
>> 
>> 
>> On Jan 11, 2012, at 1:50 PM, Jun Rao <junrao@gmail.com> wrote:
>> 
>>> Chrsitan,
>>> 
>>> A Kafka cluster containers a ZK cluster and a list of brokers. When a
>>> consumer subscribes to a topic in a kafka cluster, it consumes data
>> stored
>>> in all brokers in that cluster.
>>> 
>>> Thanks,
>>> 
>>> Jun
>>> 
>>> On Tue, Jan 10, 2012 at 11:28 PM, Christian Carollo <ccarollo@gmail.com
>>> wrote:
>>> 
>>>> Thank you Jun that is quite helpful.  I have a question about Kafka
>>>> Clusters.  What are the minimum number and types of services that must
>> be
>>>> running to make up a Kafka Cluster?
>>>> 
>>>> I ask this because the diagrams (in the Kafka Mirroring document) allude
>>>> to a multiple broker environment, however, since each broker does not
>>>> appear to provide redundancy (as of today) to any of the other brokers
>> in a
>>>> given zookeeper service, it seems like a Kafka Cluster is nothing more
>> than
>>>> a grouping of a single zookeeper instance with a single Kafka broker, is
>>>> this the correct understanding?
>>>> 
>>>> Thanks,
>>>> Christian
>>>> 
>>>> On Jan 10, 2012, at 8:47 AM, Jun Rao wrote:
>>>> 
>>>>> With 0.7, you can set up inter-cluster replication (
>>>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring).
>>>>> 
>>>>> For the future 0.8 release, we are working on intra-cluster replication
>>>>> support and details can be found at
>>>>> https://issues.apache.org/jira/browse/KAFKA-50
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Jun
>>>>> 
>>>>> On Mon, Jan 9, 2012 at 9:52 PM, Christian Carollo <ccarollo@gmail.com
>>>>> wrote:
>>>>> 
>>>>>> I am looking to implement Kafka in a production environment, however,
>> I
>>>>>> haven't found in documentation or examples that
>>>>>> discuss how to build a redundant implementation.  Is there any
>>>>>> documentation out their (blogs, articles, etc.) that describes
>>>>>> how we can implement such a system with Kafka 0.6 or 0.7.
>>>>>> 
>>>>>> Also, is there a timeframe the community is shooting for, to release
>>>> 0.8 w/
>>>>>> replication?
>>>>>> 
>>>>>> Thanks
>>>>>> Christian
>>>>>> 
>>>> 
>>>> 
>> 


Mime
View raw message