kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Kafka/ZK Cluster Example
Date Wed, 11 Jan 2012 22:53:50 GMT
It's just that the mirroring logic depends on ZK to be available most of
the time.

Jun

On Wed, Jan 11, 2012 at 2:35 PM, Christian Carollo <ccarollo@gmail.com>wrote:

> I see.  But if I used that configuration and then did the mirroring you
> suggested would that be enough, in your opinion, to be considered highly
> reliable?
>
> Christian
>
>
> On Jan 11, 2012, at 2:32 PM, Jun Rao wrote:
>
> >> For example, can I have one ZK instance and one broker on one machine
> and
> > that is enough to define a ZK cluster and a Kafka Cluster?
> >
> > Yes, although you don't get the reliability of ZK now.
> >
> > Jun
> >
> >
> > On Wed, Jan 11, 2012 at 2:06 PM, Christian Carollo <ccarollo@gmail.com
> >wrote:
> >
> >> Jun,
> >>
> >> I don't think I ask my question the right way.
> >>
> >> What I am trying to understand is what are the minimum constituent parts
> >> of a kafka cluster?
> >>
> >> Based on your last email, I am now wondering what are the minimum
> >> constituent parts of a ZK cluster as well as a Kafka cluster?
> >>
> >> For example, can I have one ZK instance and one broker on one machine
> and
> >> that is enough to define a ZK cluster and a Kafka Cluster?
> >>
> >> Thanks,
> >> Christian
> >>
> >>
> >> On Jan 11, 2012, at 1:50 PM, Jun Rao <junrao@gmail.com> wrote:
> >>
> >>> Chrsitan,
> >>>
> >>> A Kafka cluster containers a ZK cluster and a list of brokers. When a
> >>> consumer subscribes to a topic in a kafka cluster, it consumes data
> >> stored
> >>> in all brokers in that cluster.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>> On Tue, Jan 10, 2012 at 11:28 PM, Christian Carollo <
> ccarollo@gmail.com
> >>> wrote:
> >>>
> >>>> Thank you Jun that is quite helpful.  I have a question about Kafka
> >>>> Clusters.  What are the minimum number and types of services that must
> >> be
> >>>> running to make up a Kafka Cluster?
> >>>>
> >>>> I ask this because the diagrams (in the Kafka Mirroring document)
> allude
> >>>> to a multiple broker environment, however, since each broker does not
> >>>> appear to provide redundancy (as of today) to any of the other brokers
> >> in a
> >>>> given zookeeper service, it seems like a Kafka Cluster is nothing more
> >> than
> >>>> a grouping of a single zookeeper instance with a single Kafka broker,
> is
> >>>> this the correct understanding?
> >>>>
> >>>> Thanks,
> >>>> Christian
> >>>>
> >>>> On Jan 10, 2012, at 8:47 AM, Jun Rao wrote:
> >>>>
> >>>>> With 0.7, you can set up inter-cluster replication (
> >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring).
> >>>>>
> >>>>> For the future 0.8 release, we are working on intra-cluster
> replication
> >>>>> support and details can be found at
> >>>>> https://issues.apache.org/jira/browse/KAFKA-50
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Jun
> >>>>>
> >>>>> On Mon, Jan 9, 2012 at 9:52 PM, Christian Carollo <
> ccarollo@gmail.com
> >>>>> wrote:
> >>>>>
> >>>>>> I am looking to implement Kafka in a production environment,
> however,
> >> I
> >>>>>> haven't found in documentation or examples that
> >>>>>> discuss how to build a redundant implementation.  Is there any
> >>>>>> documentation out their (blogs, articles, etc.) that describes
> >>>>>> how we can implement such a system with Kafka 0.6 or 0.7.
> >>>>>>
> >>>>>> Also, is there a timeframe the community is shooting for, to
release
> >>>> 0.8 w/
> >>>>>> replication?
> >>>>>>
> >>>>>> Thanks
> >>>>>> Christian
> >>>>>>
> >>>>
> >>>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message