kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjun <ar...@socialtwist.com>
Subject Re: Delay in fetching messages - High level consumer -Kafka 0.8
Date Tue, 25 Feb 2014 18:50:53 GMT
Hi

I will make the change and see whether things work fine or not and let 
you know.

Thanks
Arjun Narasimha Kota
On Tuesday 25 February 2014 09:58 PM, Jun Rao wrote:
> The following config is probably what's causing the socket timeout. Try sth
> like 1000ms.
>
> MaxWait: 10000000 ms
>
> Thanks,
>
> Jun
>
>
> On Tue, Feb 25, 2014 at 2:16 AM, Arjun <arjun@socialtwist.com> wrote:
>
>> Apart from that i get this stack trace
>>
>> 25 Feb 2014 15:45:22,636 WARN [ConsumerFetcherThread-group1_
>> www.taf-dev.com-1393322165136-8318b07d-0-0] [kafka.consumer.ConsumerFetcherThread]
>> [ConsumerFetcherThread-group1_www.taf-dev.com-1393322165136-8318b07d-0-0],
>> Error in fetch Name: FetchRequest; Version: 0; CorrelationId: 32; ClientId:
>> group1-ConsumerFetcherThread-group1_www.taf-dev.com-1393322165136-8318b07d-0-0;
>> ReplicaId: -1; MaxWait: 10000000 ms; MinBytes: 1 bytes; RequestInfo:
>> [taf.referral.emails.service,8] -> PartitionFetchInfo(1702,1048576)
>> java.net.SocketTimeoutException
>>      at sun.nio.ch.SocketAdaptor$SocketInputStream.read(
>> SocketAdaptor.java:201)
>>      at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
>>      at java.nio.channels.Channels$ReadableByteChannelImpl.read(
>> Channels.java:221)
>>      at kafka.utils.Utils$.read(Utils.scala:395)
>>      at kafka.network.BoundedByteBufferReceive.readFrom(
>> BoundedByteBufferReceive.scala:54)
>>      at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
>>      at kafka.network.BoundedByteBufferReceive.readCompletely(
>> BoundedByteBufferReceive.scala:29)
>>      at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
>>      at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.
>> scala:81)
>>      at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$
>> $sendRequest(SimpleConsumer.scala:71)
>>      at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$
>> apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110)
>>      at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$
>> apply$mcV$sp$1.apply(SimpleConsumer.scala:110)
>>      at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$
>> apply$mcV$sp$1.apply(SimpleConsumer.scala:110)
>>      at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>>      at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(
>> SimpleConsumer.scala:109)
>>      at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(
>> SimpleConsumer.scala:109)
>>      at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(
>> SimpleConsumer.scala:109)
>>      at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>>      at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108)
>>      at kafka.server.AbstractFetcherThread.processFetchRequest(
>> AbstractFetcherThread.scala:94)
>>      at kafka.server.AbstractFetcherThread.doWork(
>> AbstractFetcherThread.scala:86)
>>      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
>>
>>
>> Does it effect anything. I havent looked at it as it was just a warning.
>> Should i be worried about this?
>>
>>
>> Thanks
>> Arjun Narasimha kota
>>
>>
>> On Tuesday 25 February 2014 03:45 PM, Arjun wrote:
>>
>>> Hi,
>>>
>>> I am using kafka 0.8. I have 3 brokers on three systems and 3 zookeepers
>>> running.
>>> I am using the high level consumer which is in examples folder of kafka.
>>> I am able to push the messages into the queue, but retriving the messages
>>> is taking some time. Is there any way i can tune this.
>>>
>>> I get this info in the kafka console, does the consumer slowness because
>>> of this??
>>> Reconnect due to socket error: null
>>>
>>> The producer is pushing the messages as i can see that using Consumer
>>> offset checker tool.I can also see there is a lag in the consumer messages
>>> in this.
>>>
>>>
>>> Thanks
>>> Arjun Narasimha Kota
>>>
>>


Mime
View raw message