metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurens Vets <laur...@daemon.be>
Subject Re: Ambari Metrics Collector failing...
Date Thu, 07 Sep 2017 15:11:09 GMT
Is this something we need to fix for 0.4.1? Also, should I create  JIRA 
ticket?

On 2017-09-06 16:45, Zeolla@GMail.com wrote:
> I'm seeing the same issue right now as well on my fresh bare metal 
> install
> of HDP (no Metron yet), haven't dug into it further to troubleshoot.
> 
> Jon
> 
> On Wed, Sep 6, 2017, 18:22 Laurens Vets <laurens@daemon.be> wrote:
> 
>> In preparation of 0.4.1-rc, I'm trying to install the current github
>> master and I'm running into an issue with Ambari-metrics-collector.
>> "Metrics Collector" seems to start, but immediately turns red again
>> Ambari and stops.
>> 
>> Any idea what might be going on or where I can start troubleshooting
>> this?
>> 
>> In /var/log/hbase-ams-master-metron1.log I see lots of:
>> 
>> 2017-09-06 21:49:16,469 INFO  [HBase-Metrics2-1] impl.MetricsConfig:
>> loaded properties from hadoop-metrics2-hbase.properties
>> 2017-09-06 21:49:16,477 INFO  [HBase-Metrics2-1]
>> timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics 
>> sink.
>> 2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1]
>> timeline.HadoopTimelineMetricsSink: Identified hostname = metron1,
>> serviceName = ams-hbase
>> 2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1]
>> timeline.HadoopTimelineMetricsSink: Collector Uri:
>> http://metron1:6188/ws/v1/timeline/metrics
>> 2017-09-06 21:49:16,491 INFO  [HBase-Metrics2-1]
>> impl.MetricsSinkAdapter: Sink timeline started
>> 2017-09-06 21:49:16,500 INFO  [HBase-Metrics2-1] 
>> impl.MetricsSystemImpl:
>> Scheduled snapshot period at 10 second(s).
>> 2017-09-06 21:49:16,500 INFO  [HBase-Metrics2-1] 
>> impl.MetricsSystemImpl:
>> HBase metrics system started
>> 2017-09-06 21:49:16,518 WARN  [HBase-Metrics2-1] lib.Interns: Metrics
>> intern cache overflow at 2011 for
>> MetricsSystem={MetricsSystem=MetricsInfo
>> Impl{name=MetricsSystem, description=MetricsSystem}, MetricsSystem
>> record=MetricsInfoImpl{name=MetricsSystem, description=MetricsSystem
>> record}}
>> 2017-09-06 21:49:17,564 WARN
>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn:
>> caught end of stream exception
>> EndOfStreamException: Unable to read additional data from client
>> sessionid 0x15e5929270d0001, likely client has closed socket
>>          at
>> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
>>          at
>> 
>> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>>          at java.lang.Thread.run(Thread.java:745)
>> 2017-09-06 21:49:17,565 INFO
>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn:
>> Closed socket connection for client /10.0.0.11:
>> 45058 which had sessionid 0x15e5929270d0001q
>> 
>> Additionally, I also see:
>> 
>> 2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
>> timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics 
>> sink.
>> 2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
>> timeline.HadoopTimelineMetricsSink: Identified hostname = metron1,
>> serviceName = ams-hbase
>> 2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
>> timeline.HadoopTimelineMetricsSink: Collector Uri:
>> http://metron1:6188/ws/v1/timeline/metrics
>> 2017-09-06 22:19:16,470 INFO  [HBase-Metrics2-1]
>> impl.MetricsSinkAdapter: Sink timeline started
>> 2017-09-06 22:19:16,471 INFO  [HBase-Metrics2-1] 
>> impl.MetricsSystemImpl:
>> Scheduled snapshot period at 10 second(s).
>> 2017-09-06 22:19:16,471 INFO  [HBase-Metrics2-1] 
>> impl.MetricsSystemImpl:
>> HBase metrics system started
>> 2017-09-06 22:20:26,491 INFO  [timeline]
>> timeline.HadoopTimelineMetricsSink: Unable to connect to collector,
>> http://metron1:6188/ws/v1/timeline/metrics
>> This exceptions will be ignored for next 100 times
>> 
>> 2017-09-06 22:20:26,491 WARN  [timeline]
>> timeline.HadoopTimelineMetricsSink: Unable to send metrics to 
>> collector
>> by address:http://metron1:6188/ws/v1/timeline/metrics
>> 
>> Which ok because metrics collector isn't working...
>> 
>> Any ideas?
>> 

Mime
View raw message