kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: doubt regarding the metadata.brokers.list parameter in producer properties
Date Thu, 19 Dec 2013 15:58:33 GMT
metadata.broker.list is used to discover brokers within the kafka cluster
on producer startup and on produce request gets error responses (that is
when the producer needs to refresh its memory of the available brokers).
While the real broker list discovered is a list of host:port, this list
used for broker discovery can be a hardware load balancer vip, etc.

Guozhang


On Thu, Dec 19, 2013 at 5:00 AM, pushkar priyadarshi <
priyadarshi.pushkar@gmail.com> wrote:

> correct
>
>
> On Thu, Dec 19, 2013 at 5:37 PM, Arjun <arjun@socialtwist.com> wrote:
>
> > So from your reply what i understood is this particular property iis used
> > only when starting the producers.
> >
> > is that right? can you please confirm.
> >
> > Thanks
> > Arjun Narasimha Kota
> >
> >
> > On Thursday 19 December 2013 05:33 PM, pushkar priyadarshi wrote:
> >
> >> 1.When you start producing : at this time if any of your supplied broker
> >> is
> >> alive system will continue to work.
> >> 2.Broker going down and coming up with new IP : producer API refreshes
> >> metadata information on failures(configurable) so they should be able to
> >> detect new brokers.
> >> But i dont think it's possible to ignore the initially supplied
> parameter.
> >>
> >>
> >> On Thu, Dec 19, 2013 at 4:57 PM, Arjun <arjun@socialtwist.com> wrote:
> >>
> >>  Hi,
> >>>
> >>> I am running kafka 0.8 and zoo keeper along with it. The problem i have
> >>> is
> >>> with the metadata.broker.list which is present in the producer
> >>> properties.
> >>> As i am using the zookeeper can i just ignore this property and kafka
> >>> will
> >>> work fine?
> >>> We run kafka on the ec2 nodes, lets say we are running some 3 kafka
> >>> servers, and producers metadata.broker.list populated with the ip's of
> >>> these nodes, tomorrow if we add two more nodes for scaling up the
> system
> >>> we
> >>> need to add those nodes to producers.
> >>> Lets say in the above scenario(my nodes ip's are non elastic), then for
> >>> some reason one of my nodes is down, and we brought up then the ip will
> >>> get
> >>> changed, so the producers should be changed to have new ip? what will
> >>> happen if one after another all these servers are down and being
> brought
> >>> up?
> >>>
> >>> Is there any way I don't give the list but kafka with the help of
> >>> zookeeper gets the list. I know kafka do not want to be dependent on
> >>> zookeeper, but what should we do in such a case.
> >>>
> >>> Thanks
> >>> Arjun Narasimha Kota
> >>>
> >>>
> >
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message