kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjkosh...@gmail.com>
Subject Re: Consumer Lag increasing but offset not moving.
Date Thu, 15 Aug 2013 00:00:38 GMT
So those are the logs for one consumer right? How many consumer
instances do you have? Can you run the offset checker tool at an
interval greater than the offset commit interval and verify that
offsets are not moving and that log sizes are growing?

https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3F

Thanks,

Joel

On Wed, Aug 14, 2013 at 4:29 PM, Ramakrishna Mallya
<rmallya@sprinklr.com> wrote:
> Hi,
>
> We are using kafka-0.7.2 with zookeeper (3.4.5)
>
> Our cluster configuration:
> 3 brokers on 3 different machines. Each broker machine has a zookeeper
> instance running as well.
>
> We are currently facing a problem with messages not being consumed by the
> kafka high level consumers. This is a dedicated topic that gets a very high
> number of messages. Consumer had no Full GCs.
>
> Api called def
> createMessageStreams(topicCountMap: Map[String,Int]): Map[String,
> List[KafkaStream[Array[Byte],Array[Byte]]]]
>
> We have already handled the exceptions at the consumers level and we have
> ensured and checked that our threads are not dying.
>
> All our consumer thread stack
>
>    java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x000000065d104ee8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:662)
>
> Kafka consumer auto commit thread stack
>
> "Kafka-consumer-autocommit-0" prio=10 tid=0x00007f99ad3a2000 nid=0x330b in
> Object.wait() [0x00007f9989084000]
>    java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
> - locked <0x0000000786bdd6c0> (a org.apache.zookeeper.ClientCnxn$Packet)
> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1264)
> at org.I0Itec.zkclient.ZkConnection.writeData(ZkConnection.java:111)
> at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:813)
> at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
> at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:809)
> at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777)
> at kafka.utils.ZkUtils$.updatePersistentPath(ZkUtils.scala:103)
> at
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:251)
> at
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:248)
> at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> at
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> at
> scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:495)
> at
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:248)
> at
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:246)
> at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> at kafka.utils.Pool$$anon$1.foreach(Pool.scala:53)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> at kafka.utils.Pool.foreach(Pool.scala:24)
> at
> kafka.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala:246)
> at
> kafka.consumer.ZookeeperConsumerConnector.autoCommit(ZookeeperConsumerConnector.scala:232)
> at
> kafka.consumer.ZookeeperConsumerConnector$$anonfun$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:126)
> at kafka.utils.Utils$$anon$2.run(Utils.scala:58)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
>
> Fetcher Runnable stack
>
> "FetchRunnable-0" prio=10 tid=0x00007f997000c000 nid=0x38b0
> sleeping[0x00007f9948da2000]
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:99)
>
>
>
>
> We have defined 300 partitions per broker, and the fetch requests is not
> processing 857 of those partitions, they are blocked, even though data is
> being added to those partitions.
>
> We are not seeing any unusual logging taking place in either kakfa brokers
> or the zookeeper clusters.
>
>
> Following is the detail of the fetcher runnable in trace mode.
>
>
> 2013-08-14 23:28:13 TRACE FetcherRunnable:35 - recevied response from fetch
> request: List(FetchRequest(topic:event_queue, part:290 offset:13858793
> maxSize:3145728), FetchRequest(topic:event_queue, part:71 offset:1919388040
> maxSize:3145728), FetchRequest(topic:event_queue, part:14 offset:1924428035
> maxSize:3145728), FetchRequest(topic:event_queue, part:287 offset:13753665
> maxSize:3145728), FetchRequest(topic:event_queue, part:21 offset:1924929744
> maxSize:3145728), FetchRequest(topic:event_queue, part:148 offset:237226086
> maxSize:3145728), FetchRequest(topic:event_queue, part:220 offset:14046778
> maxSize:3145728), FetchRequest(topic:event_queue, part:66 offset:1917452262
> maxSize:3145728), FetchRequest(topic:event_queue, part:248 offset:13820576
> maxSize:3145728), FetchRequest(topic:event_queue, part:213 offset:14009238
> maxSize:3145728), FetchRequest(topic:event_queue, part:176 offset:13889675
> maxSize:3145728), FetchRequest(topic:event_queue, part:24 offset:1922108511
> maxSize:3145728), FetchRequest(topic:event_queue, part:276 offset:13860230
> maxSize:3145728), FetchRequest(topic:event_queue, part:57 offset:1916430767
> maxSize:3145728), FetchRequest(topic:event_queue, part:73 offset:1915483219
> maxSize:3145728), FetchRequest(topic:event_queue, part:85 offset:238172050
> maxSize:3145728), FetchRequest(topic:event_queue, part:150 offset:13902607
> maxSize:3145728), FetchRequest(topic:event_queue, part:31 offset:1923158067
> maxSize:3145728), FetchRequest(topic:event_queue, part:252 offset:13880253
> maxSize:3145728), FetchRequest(topic:event_queue, part:185 offset:13632601
> maxSize:3145728), FetchRequest(topic:event_queue, part:241 offset:13870809
> maxSize:3145728), FetchRequest(topic:event_queue, part:169 offset:13653452
> maxSize:3145728), FetchRequest(topic:event_queue, part:157 offset:13698490
> maxSize:3145728), FetchRequest(topic:event_queue, part:84 offset:235590608
> maxSize:3145728), FetchRequest(topic:event_queue, part:259 offset:14340519
> maxSize:3145728), FetchRequest(topic:event_queue, part:139 offset:235498949
> maxSize:3145728), FetchRequest(topic:event_queue, part:112 offset:235697262
> maxSize:3145728), FetchRequest(topic:event_queue, part:202 offset:13646959
> maxSize:3145728), FetchRequest(topic:event_queue, part:192 offset:13743839
> maxSize:3145728), FetchRequest(topic:event_queue, part:42 offset:1922290453
> maxSize:3145728), FetchRequest(topic:event_queue, part:122 offset:235463836
> maxSize:3145728), FetchRequest(topic:event_queue, part:94 offset:236971652
> maxSize:3145728), FetchRequest(topic:event_queue, part:101 offset:235374917
> maxSize:3145728), FetchRequest(topic:event_queue, part:280 offset:13739293
> maxSize:3145728), FetchRequest(topic:event_queue, part:298 offset:14022233
> maxSize:3145728), FetchRequest(topic:event_queue, part:119 offset:235026837
> maxSize:3145728), FetchRequest(topic:event_queue, part:49 offset:1924256307
> maxSize:3145728), FetchRequest(topic:event_queue, part:168 offset:13845729
> maxSize:3145728), FetchRequest(topic:event_queue, part:23 offset:1922510880
> maxSize:3145728), FetchRequest(topic:event_queue, part:195 offset:13936005
> maxSize:3145728), FetchRequest(topic:event_queue, part:277 offset:13648569
> maxSize:3145728), FetchRequest(topic:event_queue, part:230 offset:13905437
> maxSize:3145728), FetchRequest(topic:event_queue, part:91 offset:234971350
> maxSize:3145728), FetchRequest(topic:event_queue, part:41 offset:1923355011
> maxSize:3145728), FetchRequest(topic:event_queue, part:279 offset:14000137
> maxSize:3145728), FetchRequest(topic:event_queue, part:65 offset:1914859800
> maxSize:3145728), FetchRequest(topic:event_queue, part:221 offset:13826817
> maxSize:3145728), FetchRequest(topic:event_queue, part:32 offset:1920897120
> maxSize:3145728), FetchRequest(topic:event_queue, part:251 offset:13587337
> maxSize:3145728), FetchRequest(topic:event_queue, part:177 offset:13673153
> maxSize:3145728), FetchRequest(topic:event_queue, part:288 offset:13738515
> maxSize:3145728), FetchRequest(topic:event_queue, part:212 offset:13972659
> maxSize:3145728), FetchRequest(topic:event_queue, part:249 offset:13687993
> maxSize:3145728), FetchRequest(topic:event_queue, part:270 offset:13973886
> maxSize:3145728), FetchRequest(topic:event_queue, part:158 offset:13961991
> maxSize:3145728), FetchRequest(topic:event_queue, part:260 offset:13749477
> maxSize:3145728), FetchRequest(topic:event_queue, part:13 offset:1922663539
> maxSize:3145728), FetchRequest(topic:event_queue, part:56 offset:1919536166
> maxSize:3145728), FetchRequest(topic:event_queue, part:223 offset:13634407
> maxSize:3145728), FetchRequest(topic:event_queue, part:194 offset:13726728
> maxSize:3145728), FetchRequest(topic:event_queue, part:63 offset:1920874310
> maxSize:3145728), FetchRequest(topic:event_queue, part:129 offset:238085303
> maxSize:3145728), FetchRequest(topic:event_queue, part:4 offset:1926372785
> maxSize:3145728), FetchRequest(topic:event_queue, part:120 offset:235299544
> maxSize:3145728), FetchRequest(topic:event_queue, part:111 offset:236673617
> maxSize:3145728), FetchRequest(topic:event_queue, part:36 offset:1923256197
> maxSize:3145728), FetchRequest(topic:event_queue, part:167 offset:13864323
> maxSize:3145728), FetchRequest(topic:event_queue, part:278 offset:14512652
> maxSize:3145728), FetchRequest(topic:event_queue, part:22 offset:1921366567
> maxSize:3145728), FetchRequest(topic:event_queue, part:50 offset:1921299677
> maxSize:3145728), FetchRequest(topic:event_queue, part:102 offset:235378466
> maxSize:3145728), FetchRequest(topic:event_queue, part:203 offset:13909878
> maxSize:3145728), FetchRequest(topic:event_queue, part:269 offset:13730961
> maxSize:3145728), FetchRequest(topic:event_queue, part:43 offset:1923207679
> maxSize:3145728), FetchRequest(topic:event_queue, part:29 offset:1922837141
> maxSize:3145728), FetchRequest(topic:event_queue, part:93 offset:235942183
> maxSize:3145728), FetchRequest(topic:event_queue, part:231 offset:13740564
> maxSize:3145728), FetchRequest(topic:event_queue, part:15 offset:1924837631
> maxSize:3145728), FetchRequest(topic:event_queue, part:174 offset:14167227
> maxSize:3145728), FetchRequest(topic:event_queue, part:184 offset:13792167
> maxSize:3145728), FetchRequest(topic:event_queue, part:138 offset:237532346
> maxSize:3145728), FetchRequest(topic:event_queue, part:222 offset:14037383
> maxSize:3145728), FetchRequest(topic:event_queue, part:240 offset:13699592
> maxSize:3145728), FetchRequest(topic:event_queue, part:74 offset:1915585918
> maxSize:3145728), FetchRequest(topic:event_queue, part:159 offset:13656050
> maxSize:3145728), FetchRequest(topic:event_queue, part:145 offset:235316997
> maxSize:3145728), FetchRequest(topic:event_queue, part:83 offset:239701975
> maxSize:3145728), FetchRequest(topic:event_queue, part:55 offset:1922797177
> maxSize:3145728), FetchRequest(topic:event_queue, part:297 offset:13892607
> maxSize:3145728), FetchRequest(topic:event_queue, part:186 offset:13827509
> maxSize:3145728), FetchRequest(topic:event_queue, part:152 offset:13992704
> maxSize:3145728), FetchRequest(topic:event_queue, part:257 offset:13974765
> maxSize:3145728), FetchRequest(topic:event_queue, part:130 offset:238644589
> maxSize:3145728), FetchRequest(topic:event_queue, part:89 offset:236607948
> maxSize:3145728), FetchRequest(topic:event_queue, part:264 offset:14004638
> maxSize:3145728), FetchRequest(topic:event_queue, part:64 offset:1920319726
> maxSize:3145728), FetchRequest(topic:event_queue, part:103 offset:235407571
> maxSize:3145728), FetchRequest(topic:event_queue, part:250 offset:14072244
> maxSize:3145728), FetchRequest(topic:event_queue, part:92 offset:236982568
> maxSize:3145728), FetchRequest(topic:event_queue, part:166 offset:13706618
> maxSize:3145728), FetchRequest(topic:event_queue, part:8 offset:1920802613
> maxSize:3145728), FetchRequest(topic:event_queue, part:271 offset:14091149
> maxSize:3145728), FetchRequest(topic:event_queue, part:193 offset:13919759
> maxSize:3145728), FetchRequest(topic:event_queue, part:200 offset:13964577
> maxSize:3145728), FetchRequest(topic:event_queue, part:1 offset:1921914028
> maxSize:3145728), FetchRequest(topic:event_queue, part:179 offset:13693807
> maxSize:3145728), FetchRequest(topic:event_queue, part:2 offset:1923189002
> maxSize:3145728), FetchRequest(topic:event_queue, part:160 offset:13987352
> maxSize:3145728), FetchRequest(topic:event_queue, part:82 offset:234913477
> maxSize:3145728), FetchRequest(topic:event_queue, part:96 offset:236478691
> maxSize:3145728), FetchRequest(topic:event_queue, part:175 offset:13985981
> maxSize:3145728), FetchRequest(topic:event_queue, part:30 offset:1928338809
> maxSize:3145728), FetchRequest(topic:event_queue, part:44 offset:1927344365
> maxSize:3145728), FetchRequest(topic:event_queue, part:61 offset:1917076449
> maxSize:3145728), FetchRequest(topic:event_queue, part:54 offset:1918544576
> maxSize:3145728), FetchRequest(topic:event_queue, part:137 offset:235429924
> maxSize:3145728), FetchRequest(topic:event_queue, part:146 offset:234024621
> maxSize:3145728), FetchRequest(topic:event_queue, part:68 offset:1915824648
> maxSize:3145728), FetchRequest(topic:event_queue, part:207 offset:13983596
> maxSize:3145728), FetchRequest(topic:event_queue, part:214 offset:13843173
> maxSize:3145728), FetchRequest(topic:event_queue, part:131 offset:239218119
> maxSize:3145728), FetchRequest(topic:event_queue, part:75 offset:234730864
> maxSize:3145728), FetchRequest(topic:event_queue, part:242 offset:13924920
> maxSize:3145728), FetchRequest(topic:event_queue, part:272 offset:14129560
> maxSize:3145728), FetchRequest(topic:event_queue, part:16 offset:1923561885
> maxSize:3145728), FetchRequest(topic:event_queue, part:124 offset:236152284
> maxSize:3145728), FetchRequest(topic:event_queue, part:187 offset:14249552
> maxSize:3145728), FetchRequest(topic:event_queue, part:81 offset:234178502
> maxSize:3145728), FetchRequest(topic:event_queue, part:228 offset:14033290
> maxSize:3145728), FetchRequest(topic:event_queue, part:165 offset:14133744
> maxSize:3145728), FetchRequest(topic:event_queue, part:110 offset:236192852
> maxSize:3145728), FetchRequest(topic:event_queue, part:258 offset:14139932
> maxSize:3145728), FetchRequest(topic:event_queue, part:151 offset:14008504
> maxSize:3145728), FetchRequest(topic:event_queue, part:117 offset:237862268
> maxSize:3145728), FetchRequest(topic:event_queue, part:172 offset:13823876
> maxSize:3145728), FetchRequest(topic:event_queue, part:215 offset:13807059
> maxSize:3145728), FetchRequest(topic:event_queue, part:243 offset:13579687
> maxSize:3145728), FetchRequest(topic:event_queue, part:299 offset:14130546
> maxSize:3145728), FetchRequest(topic:event_queue, part:17 offset:1924461681
> maxSize:3145728), FetchRequest(topic:event_queue, part:45 offset:1924311439
> maxSize:3145728), FetchRequest(topic:event_queue, part:132 offset:238453123
> maxSize:3145728), FetchRequest(topic:event_queue, part:285 offset:13712821
> maxSize:3145728), FetchRequest(topic:event_queue, part:154 offset:13729543
> maxSize:3145728), FetchRequest(topic:event_queue, part:229 offset:14063729
> maxSize:3145728), FetchRequest(topic:event_queue, part:201 offset:14202133
> maxSize:3145728), FetchRequest(topic:event_queue, part:161 offset:14083985
> maxSize:3145728), FetchRequest(topic:event_queue, part:62 offset:1920871581
> maxSize:3145728), FetchRequest(topic:event_queue, part:95 offset:236078056
> maxSize:3145728), FetchRequest(topic:event_queue, part:76 offset:235057152
> maxSize:3145728), FetchRequest(topic:event_queue, part:67 offset:1918530968
> maxSize:3145728), FetchRequest(topic:event_queue, part:118 offset:236009547
> maxSize:3145728), FetchRequest(topic:event_queue, part:10 offset:1920294682
> maxSize:3145728), FetchRequest(topic:event_queue, part:286 offset:14140007
> maxSize:3145728), FetchRequest(topic:event_queue, part:90 offset:235531660
> maxSize:3145728), FetchRequest(topic:event_queue, part:164 offset:13949421
> maxSize:3145728), FetchRequest(topic:event_queue, part:234 offset:14068535
> maxSize:3145728), FetchRequest(topic:event_queue, part:38 offset:1921910104
> maxSize:3145728), FetchRequest(topic:event_queue, part:181 offset:13684468
> maxSize:3145728), FetchRequest(topic:event_queue, part:7 offset:1922581694
> maxSize:3145728), FetchRequest(topic:event_queue, part:35 offset:1919656737
> maxSize:3145728), FetchRequest(topic:event_queue, part:109 offset:238515986
> maxSize:3145728), FetchRequest(topic:event_queue, part:188 offset:14030914
> maxSize:3145728), FetchRequest(topic:event_queue, part:104 offset:237177303
> maxSize:3145728), FetchRequest(topic:event_queue, part:123 offset:235236776
> maxSize:3145728), FetchRequest(topic:event_queue, part:0 offset:1921886214
> maxSize:3145728), FetchRequest(topic:event_queue, part:245 offset:13954109
> maxSize:3145728), FetchRequest(topic:event_queue, part:136 offset:236161678
> maxSize:3145728), FetchRequest(topic:event_queue, part:199 offset:14224604
> maxSize:3145728), FetchRequest(topic:event_queue, part:227 offset:14037136
> maxSize:3145728), FetchRequest(topic:event_queue, part:143 offset:235733339
> maxSize:3145728), FetchRequest(topic:event_queue, part:206 offset:14198134
> maxSize:3145728), FetchRequest(topic:event_queue, part:216 offset:13736453
> maxSize:3145728), FetchRequest(topic:event_queue, part:18 offset:1921683680
> maxSize:3145728), FetchRequest(topic:event_queue, part:237 offset:13669627
> maxSize:3145728), FetchRequest(topic:event_queue, part:126 offset:235772779
> maxSize:3145728), FetchRequest(topic:event_queue, part:209 offset:13803127
> maxSize:3145728), FetchRequest(topic:event_queue, part:294 offset:13930318
> maxSize:3145728), FetchRequest(topic:event_queue, part:153 offset:14247805
> maxSize:3145728), FetchRequest(topic:event_queue, part:59 offset:1916170361
> maxSize:3145728), FetchRequest(topic:event_queue, part:80 offset:236311660
> maxSize:3145728), FetchRequest(topic:event_queue, part:87 offset:235675888
> maxSize:3145728), FetchRequest(topic:event_queue, part:273 offset:13894364
> maxSize:3145728), FetchRequest(topic:event_queue, part:291 offset:14120121
> maxSize:3145728), FetchRequest(topic:event_queue, part:37 offset:1926437324
> maxSize:3145728), FetchRequest(topic:event_queue, part:28 offset:1923884304
> maxSize:3145728), FetchRequest(topic:event_queue, part:244 offset:13639986
> maxSize:3145728), FetchRequest(topic:event_queue, part:162 offset:13631619
> maxSize:3145728), FetchRequest(topic:event_queue, part:266 offset:14276657
> maxSize:3145728), FetchRequest(topic:event_queue, part:133 offset:234190625
> maxSize:3145728), FetchRequest(topic:event_queue, part:173 offset:13718276
> maxSize:3145728), FetchRequest(topic:event_queue, part:284 offset:13634149
> maxSize:3145728), FetchRequest(topic:event_queue, part:88 offset:239251670
> maxSize:3145728), FetchRequest(topic:event_queue, part:256 offset:13963549
> maxSize:3145728), FetchRequest(topic:event_queue, part:235 offset:14268066
> maxSize:3145728), FetchRequest(topic:event_queue, part:180 offset:13994435
> maxSize:3145728), FetchRequest(topic:event_queue, part:77 offset:237671159
> maxSize:3145728), FetchRequest(topic:event_queue, part:108 offset:236023907
> maxSize:3145728), FetchRequest(topic:event_queue, part:263 offset:13953716
> maxSize:3145728), FetchRequest(topic:event_queue, part:70 offset:1919192226
> maxSize:3145728), FetchRequest(topic:event_queue, part:142 offset:234821981
> maxSize:3145728), FetchRequest(topic:event_queue, part:105 offset:236803009
> maxSize:3145728), FetchRequest(topic:event_queue, part:156 offset:14194305
> maxSize:3145728), FetchRequest(topic:event_queue, part:144 offset:235378461
> maxSize:3145728), FetchRequest(topic:event_queue, part:46 offset:1928419689
> maxSize:3145728), FetchRequest(topic:event_queue, part:198 offset:13974447
> maxSize:3145728), FetchRequest(topic:event_queue, part:149 offset:238015271
> maxSize:3145728), FetchRequest(topic:event_queue, part:115 offset:235811382
> maxSize:3145728), FetchRequest(topic:event_queue, part:98 offset:235100030
> maxSize:3145728), FetchRequest(topic:event_queue, part:265 offset:13907349
> maxSize:3145728), FetchRequest(topic:event_queue, part:274 offset:13864687
> maxSize:3145728), FetchRequest(topic:event_queue, part:226 offset:13537827
> maxSize:3145728), FetchRequest(topic:event_queue, part:9 offset:1923454590
> maxSize:3145728), FetchRequest(topic:event_queue, part:163 offset:13882392
> maxSize:3145728), FetchRequest(topic:event_queue, part:255 offset:13946694
> maxSize:3145728), FetchRequest(topic:event_queue, part:60 offset:1922506946
> maxSize:3145728), FetchRequest(topic:event_queue, part:134 offset:236241772
> maxSize:3145728), FetchRequest(topic:event_queue, part:293 offset:14174207
> maxSize:3145728), FetchRequest(topic:event_queue, part:69 offset:1917935964
> maxSize:3145728), FetchRequest(topic:event_queue, part:53 offset:1920040686
> maxSize:3145728), FetchRequest(topic:event_queue, part:189 offset:14081801
> maxSize:3145728), FetchRequest(topic:event_queue, part:125 offset:235462272
> maxSize:3145728), FetchRequest(topic:event_queue, part:27 offset:1922583519
> maxSize:3145728), FetchRequest(topic:event_queue, part:107 offset:235043816
> maxSize:3145728), FetchRequest(topic:event_queue, part:292 offset:14225910
> maxSize:3145728), FetchRequest(topic:event_queue, part:217 offset:14155810
> maxSize:3145728), FetchRequest(topic:event_queue, part:208 offset:13869168
> maxSize:3145728), FetchRequest(topic:event_queue, part:79 offset:237019821
> maxSize:3145728), FetchRequest(topic:event_queue, part:171 offset:13921149
> maxSize:3145728), FetchRequest(topic:event_queue, part:218 offset:14012275
> maxSize:3145728), FetchRequest(topic:event_queue, part:52 offset:1920570629
> maxSize:3145728), FetchRequest(topic:event_queue, part:283 offset:13970846
> maxSize:3145728), FetchRequest(topic:event_queue, part:47 offset:1924652182
> maxSize:3145728), FetchRequest(topic:event_queue, part:232 offset:14499312
> maxSize:3145728), FetchRequest(topic:event_queue, part:211 offset:13847224
> maxSize:3145728), FetchRequest(topic:event_queue, part:141 offset:237422517
> maxSize:3145728), FetchRequest(topic:event_queue, part:106 offset:237027686
> maxSize:3145728), FetchRequest(topic:event_queue, part:26 offset:1921991622
> maxSize:3145728), FetchRequest(topic:event_queue, part:236 offset:13700792
> maxSize:3145728), FetchRequest(topic:event_queue, part:190 offset:14048595
> maxSize:3145728), FetchRequest(topic:event_queue, part:225 offset:13975669
> maxSize:3145728), FetchRequest(topic:event_queue, part:40 offset:1922256327
> maxSize:3145728), FetchRequest(topic:event_queue, part:116 offset:235790080
> maxSize:3145728), FetchRequest(topic:event_queue, part:33 offset:1922681776
> maxSize:3145728), FetchRequest(topic:event_queue, part:197 offset:13786691
> maxSize:3145728), FetchRequest(topic:event_queue, part:5 offset:1924902525
> maxSize:3145728), FetchRequest(topic:event_queue, part:178 offset:13756870
> maxSize:3145728), FetchRequest(topic:event_queue, part:204 offset:14337005
> maxSize:3145728), FetchRequest(topic:event_queue, part:97 offset:236192027
> maxSize:3145728), FetchRequest(topic:event_queue, part:239 offset:13759979
> maxSize:3145728), FetchRequest(topic:event_queue, part:78 offset:233418728
> maxSize:3145728), FetchRequest(topic:event_queue, part:121 offset:235819188
> maxSize:3145728), FetchRequest(topic:event_queue, part:12 offset:1922738965
> maxSize:3145728), FetchRequest(topic:event_queue, part:114 offset:235220219
> maxSize:3145728), FetchRequest(topic:event_queue, part:289 offset:14041944
> maxSize:3145728), FetchRequest(topic:event_queue, part:261 offset:13667852
> maxSize:3145728), FetchRequest(topic:event_queue, part:183 offset:13697688
> maxSize:3145728), FetchRequest(topic:event_queue, part:254 offset:13772026
> maxSize:3145728), FetchRequest(topic:event_queue, part:246 offset:13545843
> maxSize:3145728), FetchRequest(topic:event_queue, part:19 offset:1925964016
> maxSize:3145728), FetchRequest(topic:event_queue, part:25 offset:1924976308
> maxSize:3145728), FetchRequest(topic:event_queue, part:282 offset:13591865
> maxSize:3145728), FetchRequest(topic:event_queue, part:275 offset:13649678
> maxSize:3145728), FetchRequest(topic:event_queue, part:155 offset:13665620
> maxSize:3145728), FetchRequest(topic:event_queue, part:128 offset:235160241
> maxSize:3145728), FetchRequest(topic:event_queue, part:268 offset:13889869
> maxSize:3145728), FetchRequest(topic:event_queue, part:135 offset:235842584
> maxSize:3145728), FetchRequest(topic:event_queue, part:72 offset:1915802630
> maxSize:3145728), FetchRequest(topic:event_queue, part:296 offset:13958874
> maxSize:3145728), FetchRequest(topic:event_queue, part:39 offset:1923280812
> maxSize:3145728), FetchRequest(topic:event_queue, part:51 offset:1919445372
> maxSize:3145728), FetchRequest(topic:event_queue, part:20 offset:1926233046
> maxSize:3145728), FetchRequest(topic:event_queue, part:6 offset:1923071675
> maxSize:3145728), FetchRequest(topic:event_queue, part:34 offset:1925185060
> maxSize:3145728), FetchRequest(topic:event_queue, part:267 offset:14059111
> maxSize:3145728), FetchRequest(topic:event_queue, part:11 offset:1921467904
> maxSize:3145728), FetchRequest(topic:event_queue, part:48 offset:1924981816
> maxSize:3145728), FetchRequest(topic:event_queue, part:219 offset:13847303
> maxSize:3145728), FetchRequest(topic:event_queue, part:295 offset:13759737
> maxSize:3145728), FetchRequest(topic:event_queue, part:170 offset:13809787
> maxSize:3145728), FetchRequest(topic:event_queue, part:58 offset:1920889286
> maxSize:3145728), FetchRequest(topic:event_queue, part:205 offset:13929926
> maxSize:3145728), FetchRequest(topic:event_queue, part:247 offset:14051637
> maxSize:3145728), FetchRequest(topic:event_queue, part:224 offset:13651775
> maxSize:3145728), FetchRequest(topic:event_queue, part:147 offset:235945537
> maxSize:3145728), FetchRequest(topic:event_queue, part:233 offset:13648633
> maxSize:3145728), FetchRequest(topic:event_queue, part:100 offset:235475945
> maxSize:3145728), FetchRequest(topic:event_queue, part:86 offset:236263448
> maxSize:3145728), FetchRequest(topic:event_queue, part:113 offset:234976306
> maxSize:3145728), FetchRequest(topic:event_queue, part:99 offset:235273075
> maxSize:3145728), FetchRequest(topic:event_queue, part:210 offset:13753055
> maxSize:3145728), FetchRequest(topic:event_queue, part:127 offset:235577543
> maxSize:3145728), FetchRequest(topic:event_queue, part:262 offset:14314531
> maxSize:3145728), FetchRequest(topic:event_queue, part:182 offset:13947552
> maxSize:3145728), FetchRequest(topic:event_queue, part:191 offset:14075203
> maxSize:3145728), FetchRequest(topic:event_queue, part:238 offset:14341058
> maxSize:3145728), FetchRequest(topic:event_queue, part:253 offset:13961191
> maxSize:3145728), FetchRequest(topic:event_queue, part:3 offset:1921808922
> maxSize:3145728), FetchRequest(topic:event_queue, part:140 offset:235461115
> maxSize:3145728), FetchRequest(topic:event_queue, part:281 offset:14095956
> maxSize:3145728), FetchRequest(topic:event_queue, part:196 offset:13767437
> maxSize:3145728))
> 2013-08-14 23:28:13 TRACE FetcherRunnable:35 - fetched bytes: 0
> 2013-08-14 23:28:13 DEBUG FetcherRunnable:48 - backing off 10 ms
>
> --
> Best Regards,
> Ramakrishna Mallya

Mime
View raw message