metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zeolla@GMail.com" <zeo...@gmail.com>
Subject Re: Ambari Metrics Collector failing...
Date Thu, 07 Sep 2017 15:13:50 GMT
I wouldn't consider it a show stopper myself, happy to be persuaded
otherwise.  I'm not even 100% sure it's related to Metron.  I just put it
in Ambari's maintenance mode for now.

Jon

On Thu, Sep 7, 2017, 11:11 Laurens Vets <laurens@daemon.be> wrote:

> Is this something we need to fix for 0.4.1? Also, should I create  JIRA
> ticket?
>
> On 2017-09-06 16:45, Zeolla@GMail.com wrote:
> > I'm seeing the same issue right now as well on my fresh bare metal
> > install
> > of HDP (no Metron yet), haven't dug into it further to troubleshoot.
> >
> > Jon
> >
> > On Wed, Sep 6, 2017, 18:22 Laurens Vets <laurens@daemon.be> wrote:
> >
> >> In preparation of 0.4.1-rc, I'm trying to install the current github
> >> master and I'm running into an issue with Ambari-metrics-collector.
> >> "Metrics Collector" seems to start, but immediately turns red again
> >> Ambari and stops.
> >>
> >> Any idea what might be going on or where I can start troubleshooting
> >> this?
> >>
> >> In /var/log/hbase-ams-master-metron1.log I see lots of:
> >>
> >> 2017-09-06 21:49:16,469 INFO  [HBase-Metrics2-1] impl.MetricsConfig:
> >> loaded properties from hadoop-metrics2-hbase.properties
> >> 2017-09-06 21:49:16,477 INFO  [HBase-Metrics2-1]
> >> timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics
> >> sink.
> >> 2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1]
> >> timeline.HadoopTimelineMetricsSink: Identified hostname = metron1,
> >> serviceName = ams-hbase
> >> 2017-09-06 21:49:16,478 INFO  [HBase-Metrics2-1]
> >> timeline.HadoopTimelineMetricsSink: Collector Uri:
> >> http://metron1:6188/ws/v1/timeline/metrics
> >> 2017-09-06 21:49:16,491 INFO  [HBase-Metrics2-1]
> >> impl.MetricsSinkAdapter: Sink timeline started
> >> 2017-09-06 21:49:16,500 INFO  [HBase-Metrics2-1]
> >> impl.MetricsSystemImpl:
> >> Scheduled snapshot period at 10 second(s).
> >> 2017-09-06 21:49:16,500 INFO  [HBase-Metrics2-1]
> >> impl.MetricsSystemImpl:
> >> HBase metrics system started
> >> 2017-09-06 21:49:16,518 WARN  [HBase-Metrics2-1] lib.Interns: Metrics
> >> intern cache overflow at 2011 for
> >> MetricsSystem={MetricsSystem=MetricsInfo
> >> Impl{name=MetricsSystem, description=MetricsSystem}, MetricsSystem
> >> record=MetricsInfoImpl{name=MetricsSystem, description=MetricsSystem
> >> record}}
> >> 2017-09-06 21:49:17,564 WARN
> >> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn:
> >> caught end of stream exception
> >> EndOfStreamException: Unable to read additional data from client
> >> sessionid 0x15e5929270d0001, likely client has closed socket
> >>          at
> >> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> >>          at
> >>
> >>
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> >>          at java.lang.Thread.run(Thread.java:745)
> >> 2017-09-06 21:49:17,565 INFO
> >> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:61181] server.NIOServerCnxn:
> >> Closed socket connection for client /10.0.0.11:
> >> 45058 which had sessionid 0x15e5929270d0001q
> >>
> >> Additionally, I also see:
> >>
> >> 2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
> >> timeline.HadoopTimelineMetricsSink: Initializing Timeline metrics
> >> sink.
> >> 2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
> >> timeline.HadoopTimelineMetricsSink: Identified hostname = metron1,
> >> serviceName = ams-hbase
> >> 2017-09-06 22:19:16,468 INFO  [HBase-Metrics2-1]
> >> timeline.HadoopTimelineMetricsSink: Collector Uri:
> >> http://metron1:6188/ws/v1/timeline/metrics
> >> 2017-09-06 22:19:16,470 INFO  [HBase-Metrics2-1]
> >> impl.MetricsSinkAdapter: Sink timeline started
> >> 2017-09-06 22:19:16,471 INFO  [HBase-Metrics2-1]
> >> impl.MetricsSystemImpl:
> >> Scheduled snapshot period at 10 second(s).
> >> 2017-09-06 22:19:16,471 INFO  [HBase-Metrics2-1]
> >> impl.MetricsSystemImpl:
> >> HBase metrics system started
> >> 2017-09-06 22:20:26,491 INFO  [timeline]
> >> timeline.HadoopTimelineMetricsSink: Unable to connect to collector,
> >> http://metron1:6188/ws/v1/timeline/metrics
> >> This exceptions will be ignored for next 100 times
> >>
> >> 2017-09-06 22:20:26,491 WARN  [timeline]
> >> timeline.HadoopTimelineMetricsSink: Unable to send metrics to
> >> collector
> >> by address:http://metron1:6188/ws/v1/timeline/metrics
> >>
> >> Which ok because metrics collector isn't working...
> >>
> >> Any ideas?
> >>
>
-- 

Jon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message