kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhukar Bharti <bhartimadhu...@gmail.com>
Subject Re: 0.8.2.1 Client not able to connect with Kafka 0.10.0.1 cluster even the cluster has message format version 0.8.2.
Date Wed, 02 Nov 2016 12:53:09 GMT
Hi,

After checking we found that there was an issue with version id passed in
ConsumerMetadataRequest after setting it to
0(ConsumerMetadataRequest.currentVersion()). It started working!

Thanks!
Madhukar

On Tue, Nov 1, 2016 at 10:29 PM, Madhukar Bharti <bhartimadhukar@gmail.com>
wrote:

> Hi Ismael,
>
> Below is the stack trace from client.
>
> java.io.EOFException: Received -1 when reading from channel, socket has
>>> likely been closed.
>>
>> at kafka.utils.Utils$.read(Utils.scala:381)
>>
>> at kafka.network.BoundedByteBufferReceive.readFrom(
>>> BoundedByteBufferReceive.scala:54)
>>
>> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
>>
>> at kafka.network.BoundedByteBufferReceive.readCompletely(
>>> BoundedByteBufferReceive.scala:29)
>>
>> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:111)
>>
>>
> This we are getting while reading from BlockingChannel after sending
> ConsumerMetadataRequest.
> Sample code is here
> <https://github.com/madhukarbharti/kafka-8.2.1-test/blob/master/src/com/bharti/kafka/offset/OffsetHandler.java>
> .
>
> Regards,
> Madhukar
>
>
> On Tue, Nov 1, 2016 at 9:51 PM, Ismael Juma <ismael@juma.me.uk> wrote:
>
>> OffsetFetchRequest should have api key = 9, but for some reason your
>> broker
>> is receiving a request with api key = 10 (GroupCoordinatorRequest). Can
>> you
>> provide the stacktrace from the client as well?
>>
>> Ismael
>>
>> On Tue, Nov 1, 2016 at 12:13 PM, Madhukar Bharti <
>> bhartimadhukar@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > We have upgraded our cluster from 0.8.2.1 to 0.10.0.1. Now
>> > *log.message.format.version* is set to *0.8.2. *
>> > We are still using Kafka 0.8.2.1 jars in client side, when we tried to
>> > fetch consumer offset using BlockingChannel and OffsetFetchRequest as in
>> > example here
>> > <https://cwiki.apache.org/confluence/display/KAFKA/
>> > Committing+and+fetching+consumer+offsets+in+Kafka>,
>> > we are getting below exceptions in brokers.
>> >
>> > ERROR Closing socket for 172.19.8.200:9092-192.168.1.1:43682 because of
>> > > error (kafka.network.Processor)
>> > > kafka.network.InvalidRequestException: Error getting request for
>> apiKey:
>> > > 10 and apiVersion: 1
>> > >         at
>> > > kafka.network.RequestChannel$Request.liftedTree2$1(
>> > RequestChannel.scala:95)
>> > >         at
>> > > kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:87)
>> > >         at
>> > > kafka.network.Processor$$anonfun$processCompletedReceives$1.
>> > apply(SocketServer.scala:488)
>> > >         at
>> > > kafka.network.Processor$$anonfun$processCompletedReceives$1.
>> > apply(SocketServer.scala:483)
>> > >         at scala.collection.Iterator$clas
>> s.foreach(Iterator.scala:893)
>> > >         at scala.collection.AbstractIterator.foreach(
>> > Iterator.scala:1336)
>> > >         at
>> > > scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>> > >         at scala.collection.AbstractItera
>> ble.foreach(Iterable.scala:54)
>> > >         at
>> > > kafka.network.Processor.processCompletedReceives(SocketServe
>> r.scala:483)
>> > >         at kafka.network.Processor.run(SocketServer.scala:413)
>> > >         at java.lang.Thread.run(Thread.java:745)
>> > > Caused by: java.lang.IllegalArgumentException: Invalid version for
>> API
>> > key
>> > > 10: 1
>> > >         at
>> > > org.apache.kafka.common.protocol.ProtoUtils.schemaFor(
>> > ProtoUtils.java:31)
>> > >         at
>> > > org.apache.kafka.common.protocol.ProtoUtils.
>> > requestSchema(ProtoUtils.java:44)
>> > >         at
>> > > org.apache.kafka.common.protocol.ProtoUtils.
>> > parseRequest(ProtoUtils.java:60)
>> > >         at
>> > > org.apache.kafka.common.requests.GroupCoordinatorRequest.parse(
>> > GroupCoordinatorRequest.java:59)
>> > >         at
>> > > org.apache.kafka.common.requests.AbstractRequest.
>> > getRequest(AbstractRequest.java:54)
>> > >         at
>> > > kafka.network.RequestChannel$Request.liftedTree2$1(
>> > RequestChannel.scala:92)
>> > >         ... 10 more
>> > >
>> >
>> > But, the same code is working fine with Kafka 0.8.2.1 cluster.
>> > I am aware of some protocol changes has been made in Kafka-0.10.X.X but
>> > don't want to update our client to 0.10.0.1 as of now. Is there any way
>> > without updating client the same code will still give the consumer
>> offset.
>> >
>> >
>> >
>> >
>> > --
>> > Thanks and Regards,
>> > Madhukar Bharti
>> > Mob: 7845755539
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message