kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Morales de Frías <dmora...@paradigmatecnologico.com>
Subject Re: consumer reaction to auto-create topics
Date Fri, 07 Mar 2014 08:22:05 GMT
Thanks for your support.

I have explained the issue by using the console producer&consumer, in terms
of simplicity, but i have just found it in my application.

I will update this thread later with more tests.


Regards.




2014-03-06 18:56 GMT+01:00 Neha Narkhede <neha.narkhede@gmail.com>:

> I've seen this behavior when the broker was not functional. Basically what
> you see is that the console producer appears to have sent some messages.
> These messages may not have reached the server. Due to the bad state of the
> server, the consumer cannot get these messages and also runs into issues
> while rebalancing and issuing fetch requests for the topic.
>
> Could you try to run this from scratch (following the quickstart). Ensure
> that the broker and zookeeper have been started successfully.
>
> Thanks,
> Neha
>
>
> On Thu, Mar 6, 2014 at 9:34 AM, David Morales de Frías <
> dmorales@paradigmatecnologico.com> wrote:
>
> > 0.8, thanks.
> >
> >
> > 2014-03-06 18:27 GMT+01:00 Jun Rao <junrao@gmail.com>:
> >
> > > Which version of Kafka are you using?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Thu, Mar 6, 2014 at 2:10 AM, David Morales de Frías <
> > > dmorales@paradigmatecnologico.com> wrote:
> > >
> > > > Hi there,
> > > >
> > > > If i start a consumer in a non-existent topic (auto-create true)
> before
> > > the
> > > > producer, the consumer never gets the messages.
> > > >
> > > > These are the steps:
> > > >
> > > > 1) kafka-console-consumer --topic newTopic (it doesn't exist)
> > > > 2) kafka-console-producer --topic newTopic
> > > > 3) Send some messages
> > > > 4) I can see this in the log and the consumer never receives them
> > > >
> > > > [2014-03-06 10:58:18,641] WARN Error while fetching metadata
> > > > [{TopicMetadata for topic newTopic ->
> > > > No partition metadata for topic newTopic due to
> > > > kafka.common.LeaderNotAvailableException}] for topic [newTopic]:
> class
> > > > kafka.common.LeaderNotAvailableException
> > > >  (kafka.producer.BrokerPartitionInfo)
> > > >
> > > >
> > > >
> > > > 5) restart kafka-console-consumer --topic newTopic
> > > > 6) Send some messages from producer
> > > > 7) Consumer receives messages.
> > > >
> > > >
> > > >
> > > > Is that ok?
> > > > Where are the produced messages? I'm sure that consumer never see
> them,
> > > but
> > > > are they stored in brokers?
> > > >
> > > >
> > > > Regards.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message