kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjun <ar...@socialtwist.com>
Subject Re: Delay in fetching messages - High level consumer -Kafka 0.8
Date Tue, 25 Feb 2014 12:15:32 GMT
Adding to this, i have started my logs in trace mode. I fount that the 
Consumer fetcher threads are sending the meta data but are not receiving 
any.
I see all the
"TRACE 
[ConsumerFetcherThread-group1_www.taf-dev.com-1393329622308-6e15dd12-0-1] [kafka.network.BoundedByteBufferSend]

205 bytes written"
but no reading is taking place. I may be looking at some thing wrong.

Can some one please help me out in this.

Thanks
Arjun Narasimha kota


On Tuesday 25 February 2014 03:46 PM, Arjun wrote:
> Apart from that i get this stack trace
>
> 25 Feb 2014 15:45:22,636 WARN 
> [ConsumerFetcherThread-group1_www.taf-dev.com-1393322165136-8318b07d-0-0] 
> [kafka.consumer.ConsumerFetcherThread] 
> [ConsumerFetcherThread-group1_www.taf-dev.com-1393322165136-8318b07d-0-0], 
> Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 32; 
> ClientId: 
> group1-ConsumerFetcherThread-group1_www.taf-dev.com-1393322165136-8318b07d-0-0; 
> ReplicaId: -1; MaxWait: 10000000 ms; MinBytes: 1 bytes; RequestInfo: 
> [taf.referral.emails.service,8] -> PartitionFetchInfo(1702,1048576)
> java.net.SocketTimeoutException
>     at 
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
>     at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
>     at 
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:221)
>     at kafka.utils.Utils$.read(Utils.scala:395)
>     at 
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>     at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
>     at 
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>     at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
>     at 
> kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:81)
>     at 
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71)
>     at 
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110)
>     at 
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)
>     at 
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)
>     at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>     at 
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109)
>     at 
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)
>     at 
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)
>     at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>     at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108)
>     at 
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:94)
>     at 
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86)
>     at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
>
>
> Does it effect anything. I havent looked at it as it was just a 
> warning. Should i be worried about this?
>
>
> Thanks
> Arjun Narasimha kota
>
> On Tuesday 25 February 2014 03:45 PM, Arjun wrote:
>> Hi,
>>
>> I am using kafka 0.8. I have 3 brokers on three systems and 3 
>> zookeepers running.
>> I am using the high level consumer which is in examples folder of 
>> kafka. I am able to push the messages into the queue, but retriving 
>> the messages is taking some time. Is there any way i can tune this.
>>
>> I get this info in the kafka console, does the consumer slowness 
>> because of this??
>> Reconnect due to socket error: null
>>
>> The producer is pushing the messages as i can see that using Consumer 
>> offset checker tool.I can also see there is a lag in the consumer 
>> messages in this.
>>
>>
>> Thanks
>> Arjun Narasimha Kota
>


Mime
View raw message