kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pravin kumar <pk007...@gmail.com>
Subject Fwd: Consumer slowness issue
Date Tue, 27 Mar 2018 17:48:05 GMT
---------- Forwarded message ----------
From: "pravin kumar" <pk007790@gmail.com>
Date: 27-Mar-2018 6:11 PM
Subject: Consumer slowness issue
To: <users@kafka.apache.org>
Cc:

i have two topics with 5 partitions each

wikifeedInputT10

KafkaProducer produces 100000 elements and wikifeedInputT10 have
received these elements.
[admin@nms-181 bin]$ sh kafka-run-class.sh kafka.tools.GetOffsetShell
--broker-list localhost:9092 --topic wikifeedInputT10 --time -1

wikifeedInputT10:2:20000
wikifeedInputT10:4:20000
wikifeedInputT10:1:20000
wikifeedInputT10:3:20000
wikifeedInputT10:0:20000

but after processing reading from my outputTopic: wikifeedOutputT15
i have received


[admin@nms-181 bin]$ sh kafka-run-class.sh kafka.tools.GetOffsetShell
--broker-list localhost:9092 --topic wikifeedOutputT15 --time -1
wikifeedOutputT15:2:1
wikifeedOutputT15:4:1
wikifeedOutputT15:1:3
wikifeedOutputT15:3:3
wikifeedOutputT15:0:3

I have received the output in my console as

[2018-03-27 17:55:32,359] INFO Kafka version : 1.0.1
(org.apache.kafka.common.utils.AppInfoParser)
[2018-03-27 17:55:32,359] INFO Kafka commitId : c0518aa65f25317e
(org.apache.kafka.common.utils.AppInfoParser)
[2018-03-27 17:55:32,600] INFO [Consumer clientId=C2,
groupId=ConsumerWikiFeedLambda4] Discovered group coordinator
nms-181.nmsworks.co.in:9092 (id: 2147483647 rack: null)
(org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2018-03-27 17:55:32,602] INFO [Consumer clientId=C2,
groupId=ConsumerWikiFeedLambda4] Revoking previously assigned
partitions [] (org.apache.kafka.clients.consumer.internals.
ConsumerCoordinator)
[2018-03-27 17:55:32,602] INFO [Consumer clientId=C2,
groupId=ConsumerWikiFeedLambda4] (Re-)joining group
(org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
[2018-03-27 17:55:32,610] INFO [Consumer clientId=C2,
groupId=ConsumerWikiFeedLambda4] Successfully joined group with
generation 5 (org.apache.kafka.clients.consumer.internals.
AbstractCoordinator)
[2018-03-27 17:55:32,611] INFO [Consumer clientId=C2,
groupId=ConsumerWikiFeedLambda4] Setting newly assigned partitions
[wikifeedOutputT15-2, wikifeedOutputT15-1, wikifeedOutputT15-0,
wikifeedOutputT15-4, wikifeedOutputT15-3]
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
Topic :::::::wikifeedOutputT15 Partition:::::0 Key::::joe  =  Value::::::
18263
Topic :::::::wikifeedOutputT15 Partition:::::0 Key::::phil  =  Value::::::
18230
Topic :::::::wikifeedOutputT15 Partition:::::0 Key::::tania  =
Value:::::: 18344
Topic :::::::wikifeedOutputT15 Partition:::::1 Key::::pravin  =
Value:::::: 18140
Topic :::::::wikifeedOutputT15 Partition:::::1 Key::::kumar  =
Value:::::: 18248
Topic :::::::wikifeedOutputT15 Partition:::::1 Key::::joseph  =
Value:::::: 18116
Topic :::::::wikifeedOutputT15 Partition:::::2 Key::::lauren  =
Value:::::: 18150
Topic :::::::wikifeedOutputT15 Partition:::::3 Key::::bob  =  Value::::::
18131
Topic :::::::wikifeedOutputT15 Partition:::::3 Key::::erica  =
Value:::::: 18084
Topic :::::::wikifeedOutputT15 Partition:::::3 Key::::damian  =
Value:::::: 18126
Topic :::::::wikifeedOutputT15 Partition:::::4 Key::::sam  =  Value::::::
18168


it stops here and im not getting any msgs

i have attached my code below

Mime
View raw message