kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig Ching <craigch...@gmail.com>
Subject Re: Problem consuming from broker 1.1.0
Date Thu, 14 Jun 2018 04:18:19 GMT
Hi Manikumar!

Thanks for responding!  Sorry it took me so long to get back!

We did end up turning on debug logs for the console consumer and found that one broker seemed
to be having problems, it would lead to timeouts communicating with it.  After restarting
that broker, things sorted themselves out. However, I always hate the “turn off/ turn on”
solution ;)  It’s interesting to me that the 1.1.0 consumer, though it reported timeouts
in the logs, never had a problem, it seemed able to recover.  Whereas the 1.0.1 consumer (talking
to a 1.1.0 cluster remember) couldn’t recover.  Does any of this make sense?  I’m happy
to provide more details and logs if necessary as I’d like to understand the root problem
here.

Thanks again!

Cheers,
Craig

> On Jun 13, 2018, at 12:43 AM, Manikumar <manikumar.reddy@gmail.com> wrote:
> 
> Can you post consumer debug logs?
> You can enable console consumer debug logs here:
> kafka/config/tools-log4j.properties
> 
> On Wed, Jun 13, 2018 at 9:55 AM Craig Ching <craigching@gmail.com> wrote:
> 
>> Hi!
>> 
>> We’re having a problem with a new kafka cluster at 1.1.0.  The problem is,
>> in general, that consumers can’t consume from the different broker (old
>> broker was 0.11 I think).  The easiest recipe I have for reproducing the
>> problem is that downloading kafka 1.0.1 and running console consumer can’t
>> consume from the 1.1.0 cluster while a 1.1.0 console consumer can.  We’re
>> invoking console consumer like this:
>> 
>> bin/kafka-console-consumer.sh \
>>    —bootstrap-server [kafka server] \
>>    —topic [our topic] \
>>    —max-messages 3
>> 
>> That works for a 1.1.0 console consumer, but not for 1.0.1.  However, if
>> we change that to:
>> 
>> bin/kafka-console-consumer.sh \
>>    —zookeeper [zookeeper server] \
>>    —topic [our topic] \
>>    —max-messages 3
>> 
>> Then it works for 1.0.1.
>> 
>> I was wondering, is the zookeeper schema published for 1.1.0?  I have a
>> feeling that maybe something is wrong in zookeeper and I know earlier
>> versions of kafka used to publish the zk schema, could there be a problem
>> in zk?  if so, what might I look for?
>> 
>> Any help is greatly appreciated!
>> 
>> Cheers,
>> Craig


Mime
View raw message