kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig Ching <craigch...@gmail.com>
Subject Re: Problem consuming from broker 1.1.0
Date Sat, 21 Jul 2018 20:37:15 GMT
Hi Senthil,

We never did get to the root cause unfortunately.  Our kafka cluster is
currently owned by a different team and it wasn't a priority for them to
figure that out, or at least communicate the problem to us :(

Cheers,
Craig

On Thu, Jul 19, 2018 at 8:13 AM SenthilKumar K <senthilec566@gmail.com>
wrote:

> Hi Craig Ching,
>
> Reg. *We did end up turning on debug logs for the console consumer and
> found that one broker seemed to be having problems, it would lead to
> timeouts communicating with it.  After restarting that broker, things
> sorted themselves out.*
>
> We had similar problem on prod cluster and i'm trying to figure out the
> root cause for why broker stopped responding ? Pls check my email subject :
> *"**Kafka Broker Not Responding" *where i described problem in detail.
>
> Curious to know , Were you able to figure out reason for broker failure ?.
> Of Course , turning off/on is not the ideal solution.
>
> --Senthil
>
>
> On Thu, Jun 14, 2018 at 9:48 AM, Craig Ching <craigching@gmail.com> wrote:
>
> > Hi Manikumar!
> >
> > Thanks for responding!  Sorry it took me so long to get back!
> >
> > We did end up turning on debug logs for the console consumer and found
> > that one broker seemed to be having problems, it would lead to timeouts
> > communicating with it.  After restarting that broker, things sorted
> > themselves out. However, I always hate the “turn off/ turn on” solution
> ;)
> > It’s interesting to me that the 1.1.0 consumer, though it reported
> timeouts
> > in the logs, never had a problem, it seemed able to recover.  Whereas the
> > 1.0.1 consumer (talking to a 1.1.0 cluster remember) couldn’t recover.
> > Does any of this make sense?  I’m happy to provide more details and logs
> if
> > necessary as I’d like to understand the root problem here.
> >
> > Thanks again!
> >
> > Cheers,
> > Craig
> >
> > > On Jun 13, 2018, at 12:43 AM, Manikumar <manikumar.reddy@gmail.com>
> > wrote:
> > >
> > > Can you post consumer debug logs?
> > > You can enable console consumer debug logs here:
> > > kafka/config/tools-log4j.properties
> > >
> > > On Wed, Jun 13, 2018 at 9:55 AM Craig Ching <craigching@gmail.com>
> > wrote:
> > >
> > >> Hi!
> > >>
> > >> We’re having a problem with a new kafka cluster at 1.1.0.  The problem
> > is,
> > >> in general, that consumers can’t consume from the different broker
> (old
> > >> broker was 0.11 I think).  The easiest recipe I have for reproducing
> the
> > >> problem is that downloading kafka 1.0.1 and running console consumer
> > can’t
> > >> consume from the 1.1.0 cluster while a 1.1.0 console consumer can.
> > We’re
> > >> invoking console consumer like this:
> > >>
> > >> bin/kafka-console-consumer.sh \
> > >>    —bootstrap-server [kafka server] \
> > >>    —topic [our topic] \
> > >>    —max-messages 3
> > >>
> > >> That works for a 1.1.0 console consumer, but not for 1.0.1.  However,
> if
> > >> we change that to:
> > >>
> > >> bin/kafka-console-consumer.sh \
> > >>    —zookeeper [zookeeper server] \
> > >>    —topic [our topic] \
> > >>    —max-messages 3
> > >>
> > >> Then it works for 1.0.1.
> > >>
> > >> I was wondering, is the zookeeper schema published for 1.1.0?  I have
> a
> > >> feeling that maybe something is wrong in zookeeper and I know earlier
> > >> versions of kafka used to publish the zk schema, could there be a
> > problem
> > >> in zk?  if so, what might I look for?
> > >>
> > >> Any help is greatly appreciated!
> > >>
> > >> Cheers,
> > >> Craig
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message