metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurens Vets <laur...@daemon.be>
Subject Ambari Metrics Collector failing...
Date Wed, 06 Sep 2017 22:21:41 GMT
In preparation of 0.4.1-rc, I'm trying to install the current github 
master and I'm running into an issue with Ambari-metrics-collector. 
"Metrics Collector" seems to start, but immediately turns red again 
Ambari and stops.

Any idea what might be going on or where I can start troubleshooting 
this?

In /var/log/hbase-ams-master-metron1.log I see lots of:

2017-09-06 21:49:16,469 INFO  [HBase-Metrics2-1] impl.MetricsConfig: 
loaded properties from hadoop-metrics2-hbase.properties
2017-09-06 21:49:16,477 INFO  [HBase-Metrics2-1] 
timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics sink.
2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1] 
timeline.HadoopTimelineMetricsSink: Identified hostname = metron1, 
serviceName = ams-hbase
2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1] 
timeline.HadoopTimelineMetricsSink: Collector Uri: 
http://metron1:6188/ws/v1/timeline/metrics
2017-09-06 21:49:16,491 INFO  [HBase-Metrics2-1] 
impl.MetricsSinkAdapter: Sink timeline started
2017-09-06 21:49:16,500 INFO  [HBase-Metrics2-1] impl.MetricsSystemImpl: 
Scheduled snapshot period at 10 second(s).
2017-09-06 21:49:16,500 INFO  [HBase-Metrics2-1] impl.MetricsSystemImpl: 
HBase metrics system started
2017-09-06 21:49:16,518 WARN  [HBase-Metrics2-1] lib.Interns: Metrics 
intern cache overflow at 2011 for 
MetricsSystem={MetricsSystem=MetricsInfo
Impl{name=MetricsSystem, description=MetricsSystem}, MetricsSystem 
record=MetricsInfoImpl{name=MetricsSystem, description=MetricsSystem 
record}}
2017-09-06 21:49:17,564 WARN  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn: 
caught end of stream exception
EndOfStreamException: Unable to read additional data from client 
sessionid 0x15e5929270d0001, likely client has closed socket
         at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
         at java.lang.Thread.run(Thread.java:745)
2017-09-06 21:49:17,565 INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn: 
Closed socket connection for client /10.0.0.11:
45058 which had sessionid 0x15e5929270d0001q

Additionally, I also see:

2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1] 
timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics sink.
2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1] 
timeline.HadoopTimelineMetricsSink: Identified hostname = metron1, 
serviceName = ams-hbase
2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1] 
timeline.HadoopTimelineMetricsSink: Collector Uri: 
http://metron1:6188/ws/v1/timeline/metrics
2017-09-06 22:19:16,470 INFO  [HBase-Metrics2-1] 
impl.MetricsSinkAdapter: Sink timeline started
2017-09-06 22:19:16,471 INFO  [HBase-Metrics2-1] impl.MetricsSystemImpl: 
Scheduled snapshot period at 10 second(s).
2017-09-06 22:19:16,471 INFO  [HBase-Metrics2-1] impl.MetricsSystemImpl: 
HBase metrics system started
2017-09-06 22:20:26,491 INFO  [timeline] 
timeline.HadoopTimelineMetricsSink: Unable to connect to collector, 
http://metron1:6188/ws/v1/timeline/metrics
This exceptions will be ignored for next 100 times

2017-09-06 22:20:26,491 WARN  [timeline] 
timeline.HadoopTimelineMetricsSink: Unable to send metrics to collector 
by address:http://metron1:6188/ws/v1/timeline/metrics

Which ok because metrics collector isn't working...

Any ideas?

Mime
View raw message