kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: consumer not consuming messages
Date Fri, 11 Apr 2014 03:23:44 GMT
Are you using the high-level consumer? How did you set fetch.wait.max.msand
fetch.min.bytes?

Thanks,

Jun


On Thu, Apr 10, 2014 at 8:13 PM, Arjun <arjun@socialtwist.com> wrote:

> This just dosent happen only when the topic is newly created, happens even
> if topic has lot of messgaes, but all messages are consumed by the
> consumer. Now if you add just one message, consumer will not fetch, if at
> that scenario we add more than 10 messages things work fine. (10 is just a
> arbitary number)
>
> Thanks
> Arjun Narasimha Kota
>
>
> On Friday 11 April 2014 06:23 AM, Arjun Kota wrote:
>
>>
>> The consumer uses do specific topics.
>>
>> On Apr 11, 2014 6:23 AM, "Arjun Kota" <arjun@socialtwist.com <mailto:
>> arjun@socialtwist.com>> wrote:
>>
>>     Yes the message shows up on the server.
>>
>>     On Apr 11, 2014 12:07 AM, "Guozhang Wang" <wangguoz@gmail.com
>>     <mailto:wangguoz@gmail.com>> wrote:
>>
>>         Hi Arjun,
>>
>>         If you only send one message, does that message show up on the
>>         server? Does
>>         you consumer use wildcard topics or specific topics?
>>
>>         Guozhang
>>
>>
>>         On Thu, Apr 10, 2014 at 9:20 AM, Arjun <arjun@socialtwist.com
>>         <mailto:arjun@socialtwist.com>> wrote:
>>
>>         > But  we have auto offset reset to smallest not largest, even
>>         then this
>>         > issue arises? If so is there any work around?
>>         >
>>         > Thanks
>>         > Arjun NArasimha Kota
>>         >
>>         >
>>         > On Thursday 10 April 2014 09:39 PM, Guozhang Wang wrote:
>>         >
>>         >> It could be https://issues.apache.org/jira/browse/KAFKA-1006.
>>         >>
>>         >> Guozhang
>>         >>
>>         >>
>>         >> On Thu, Apr 10, 2014 at 8:50 AM, Arjun
>>         <arjun@socialtwist.com <mailto:arjun@socialtwist.com>> wrote:
>>         >>
>>         >>  its auto created
>>         >>> but even after topic creation this is the scenario
>>         >>>
>>         >>> Arjun
>>         >>>
>>         >>> On Thursday 10 April 2014 08:41 PM, Guozhang Wang wrote:
>>         >>>
>>         >>>  Hi Arjun,
>>         >>>>
>>         >>>> Did you manually create the topic or use auto.topic.creation?
>>         >>>>
>>         >>>> Guozhang
>>         >>>>
>>         >>>>
>>         >>>> On Thu, Apr 10, 2014 at 7:39 AM, Arjun
>>         <arjun@socialtwist.com <mailto:arjun@socialtwist.com>> wrote:
>>         >>>>
>>         >>>>   Hi,
>>         >>>>
>>         >>>>> We have 3 node kafka 0.8 setup with zookeepers ensemble.
>>         We use high
>>         >>>>> level
>>         >>>>> consumer with auto commit offset false. I am facing
some
>>         peculiar
>>         >>>>> problem
>>         >>>>> with kafka. When i send some 10-20 messages or so the
>>         consumer starts
>>         >>>>> to
>>         >>>>> consume the messages. But if  i send only one message
to
>>         kafka, then
>>         >>>>> even
>>         >>>>> though consumer is active it is not trying to fetch
the
>>         message. There
>>         >>>>> is
>>         >>>>> nothing in logs, just the messages are being fetched
by
>>         the kafka
>>         >>>>> consumer.
>>         >>>>> The messages are there in the Kafka server. Can some
one
>>         let me know
>>         >>>>> where
>>         >>>>> i am doing wrong.
>>         >>>>>
>>         >>>>>
>>         >>>>> Thanks
>>         >>>>> Arjun Narasimha Kota
>>         >>>>>
>>         >>>>>
>>         >>>>>
>>         >>>>
>>         >>
>>         >
>>
>>
>>         --
>>         -- Guozhang
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message