kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cosmin Marginean <cosmargin...@gmail.com>
Subject Re: Leader not available (initially)
Date Fri, 08 Jan 2016 16:51:34 GMT
Hi Dana

I am actually creating the topic manually from our code. This is mainly because I couldn’t
tell from the documentation when exactly is the topic auto-created - maybe you can shed some
light on that.

The docs (http://kafka.apache.org/documentation.html) only mention
auto.create.topics.enable Enable auto creation of topic on the server
which is a bit without context.

Either way (automated or not), it is crucial for us to know if the message actually gets sent.
The problem is we need to get the producer to send some messages as soon as the system bootstraps
(and potentially creates the topic). So is it possible to know (listener, etc) on the producer
side when the topic is “operational”, or at least to handle a specific exception?

Or is it a case where the producer will buffer these on the client side?

Thanks
Cos



On Friday, 8 January 2016 at 16:45, Dana Powers wrote:

> That is expected when a topic is first created. Once the topic and all
> partitions are initialized, the leader(s) will be available and you can
> produce messages.
>  
> Note that if you disable topic auto creation, you should get an
> UnknownTopicOrPartitionError, and you would need to create the topic
> manually.
>  
> -Dana
> On Jan 8, 2016 03:20, "Cosmin Marginean" <cosmarginean@gmail.com (mailto:cosmarginean@gmail.com)>
wrote:
>  
> > Hi guys
> >  
> > I’ve been seeing this in our logs a few times, when initially
> > bootstrapping the system.
> >  
> > WARN 2016-01-08 10:59:24,232 [pool-3-thread-11] [none]
> > o.a.kafka.clients.NetworkClient: Error while fetching metadata with
> > correlation id 3 : {mytopic=LEADER_NOT_AVAILABLE}
> >  
> > It doesn’t seem to break anything, and it seems to be related to the fact
> > that we do processing on the topic right after creating it. Below is our
> > workflow (roughly)
> > * Create topic (using AdminUtils)
> > * Register some tens of consumers
> > * Send some messages with a producer
> >  
> > I’d like to know if this is something to worry about (the warning that is)
> > or if it’s something we should investigate further.
> >  
> > Thank you
> > Cosmin
> >  
>  
>  
>  



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message