storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyank Shah <ps...@hortonworks.com>
Subject Re: Kafka monitor unable to get offset lag
Date Wed, 01 Feb 2017 19:28:15 GMT
Hi Igor,

Just to clarify, are you saying that running storm-kafka-monitor from command line with the
arguments you pasted returns the spout lags correctly ? Also, can you confirm if you are running
in a secured setup or not ?

From: Igor Kuzmenko <f1sherox@gmail.com>
Reply-To: "user@storm.apache.org" <user@storm.apache.org>
Date: Wednesday, February 1, 2017 at 5:05 AM
To: "user@storm.apache.org" <user@storm.apache.org>
Subject: Re: Kafka monitor unable to get offset lag

Found in logs with debug level:
2017-02-01 14:06:54.053 o.a.s.u.TopologySpoutLag [DEBUG] spout classname: org.apache.storm.kafka.spout.KafkaSpout
2017-02-01 14:06:54.053 o.a.s.u.TopologySpoutLag [DEBUG] json configuration: {config.bootstrap.servers=dn-06.ru:6667<http://dn-06.ru:6667>,dn-10.ru:6667<http://dn-10.ru:6667>,
config.topics=gtp,, config.groupid=test, topology.tasks=1}
2017-02-01 14:06:54.053 o.a.s.u.TopologySpoutLag [DEBUG] Command to run: [/usr/hdp/2.5.0.0-1245/storm/bin/storm-kafka-monitor,
-t, gtp,, -g, test, -b, dn-06.ru:6667<http://dn-06.ru:6667>,dn-10.ru:6667<http://dn-10.ru:6667>]
2017-02-01 14:07:36.338 o.a.s.u.TopologySpoutLag [DEBUG] JSON parsing failed, assuming message
as error message: Unable to get offset lags for kafka. Reason: org.apache.kafka.shaded.common.errors.TimeoutException:
Timeout expired while fetching topic metadata

Running command /usr/hdp/2.5.0.0-1245/storm/bin/storm-kafka-monitor -t gtp -g test -b dn-06.ru:6667<http://dn-06.ru:6667>,dn-10.ru:6667<http://dn-10.ru:6667>
 returns correct json with spout lags. There's no timeout command executing in a second.



On Wed, Feb 1, 2017 at 12:25 PM, Spico Florin <spicoflorin@gmail.com<mailto:spicoflorin@gmail.com>>
wrote:
Hello!
  You can check how your topic is consumed  and its health via the scripts:
./kafka-consumer-groups.sh --new-consumer --bootstrap-server <your-broker>:6667 --list
 ./kafka-consumer-groups.sh --new-consumer --bootstrap-server <your-broker>:6667 --describe
--group <your-group>

I hope it helps.
 Florin

On Wed, Feb 1, 2017 at 11:01 AM, Igor Kuzmenko <f1sherox@gmail.com<mailto:f1sherox@gmail.com>>
wrote:
Yes, topology process data and works fine.
I couldn't find any exceptions in storm logs.   access-web-ui.log contains only these lines

2017-02-01 11:58:17.315 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: 10.35.63.14 url:
http://master001.s:8744/api/v1/cluster/summary principal:
2017-02-01 11:58:17.315 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: 10.35.63.14 url:
http://master001.s:8744/api/v1/nimbus/summary principal:
2017-02-01 11:58:25.346 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: 10.35.63.14 url:
http://master001.s:8744/api/v1/topology/summary principal:
2017-02-01 11:58:25.346 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: 10.35.63.14 url:
http://master001.s:8744/api/v1/cluster/summary principal:
2017-02-01 11:58:25.346 o.a.s.l.f.AccessLoggingFilter [INFO] Access from: 10.35.63.14 url:
http://master001.s:8744/api/v1/nimbus/summary principal:

On Tue, Jan 31, 2017 at 5:19 PM, Priyank Shah <pshah@hortonworks.com<mailto:pshah@hortonworks.com>>
wrote:
Hi Igor,

When you say topology is working fine do you mean you see data flowing? Can you try to look
up the logs for ui server and paste relevant lines here if any?

Priyank

Sent from my iPhone

On Jan 31, 2017, at 4:34 AM, Igor Kuzmenko <f1sherox@gmail.com<mailto:f1sherox@gmail.com>>
wrote:
I've launched topology with new kafka spout. Topology by it self working fine, but looking
at storm UI I see kafka-monitor exception:
Unable to get offset lags for kafka. Reason: org.apache.kafka.shaded.common.errors.TimeoutException:
Timeout expired while fetching topic metadata

Maybe I forgot to configure something, but then how topology reads messages?



Mime
View raw message