nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Omer Hadari <hadari.o...@gmail.com>
Subject Re: Status of MetricsReportingTask?
Date Fri, 02 Nov 2018 22:38:32 GMT
Hi Jon. Just wanted to say that I have developed this ReportingTask since
we had a similar issue to yours, only we had to report metrics to graphite.
We are currently using it in production.
I tried to implement as generic design as possible, so reporting to other
endpoints (e.g. cloudwatch) should be easy enough.
I performed a quick google search about dropwizard-metrics cloudwatch
reporter implementation. Found many small projects, such as this one:
https://github.com/azagniotov/codahale-aggregated-metrics-cloudwatch-reporter.
It might fit your needs or you can at least take a look at it.

When I designed this I only had system and process group metrics in mind. I
was trying to be generic with the endpoints, so I used dropwizard-metrics
but I admit I haven't thought about allowing custom tasks to report custom
metrics. It shouldn't be too hard to implement even now, so let me know if
you need this option and I will try getting to submitting a contribution.
Feel free to let me know if you have any questions regarding this reporting
task.

On Fri, Nov 2, 2018 at 9:35 PM Bryan Bende <bbende@gmail.com> wrote:

> You should be able to make your own NAR with two modules like:
>
> nifi-cloudwatch-metrics - contains your service implementation with a
> provided dependency on nifi-metrics-reporter-service-api
> nifi-cloudwatch-metrics-nar - packages your implementation and has a
> NAR dependency on nifi-metrics-reporter-service-api-nar
>
> Similar to this example:
>
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-LinkingProcessorsandControllerServices
>
> On Fri, Nov 2, 2018 at 3:24 PM Jon Logan <jmlogan@buffalo.edu> wrote:
> >
> > We were planning on having a reporter to pipe the metrics off to AWS
> > CloudWatch. The one issue being ran into -- and we've ran into it for
> other
> > components -- is the classloading. What is the recommended way of adding
> > alternate implementations of services (in this case
> MetricReporterService)?
> > We run into issues that I think stem from classloading isolation between
> > NARs, such that our custom implementations of services are not being
> > detected from the other NAR's context. We end up having to duplicate the
> > service into our custom NAR, but that hardly seems like a good idea.
> >
> > Thanks!
> > Jon
> >
> > On Thu, Nov 1, 2018 at 10:27 PM Koji Kawamura <ijokarumawak@gmail.com>
> > wrote:
> >
> > > Hi Jon,
> > >
> > > About reporting counter values, there is an existing JIRA for the
> > > improvement idea to expose counters to Reporting task context. That
> > > requires NiFi framework level improvements. I'd suggest taking a look
> > > on it, and resuming discussion there if needed.
> > > https://issues.apache.org/jira/browse/NIFI-3293
> > >
> > > Although MetricsReportingTask currently only supports Graphite, the
> > > component is well designed for generic reporting use-cases. I don't
> > > think it is a legacy component. The underlying dropwizard metrics
> > > project seems active, too.
> > > https://github.com/dropwizard/metrics
> > >
> > > I'm interested in what service implementation you're going to write.
> > > Is it going to use one of already available reporters?
> > > https://metrics.dropwizard.io/3.1.0/manual/third-party/
> > >
> > > Thanks,
> > > Koji
> > > On Thu, Nov 1, 2018 at 3:17 AM Jon Logan <jmlogan@buffalo.edu> wrote:
> > > >
> > > > Hi All,
> > > >
> > > > I was looking at utilizing the MetricsReportingTask service, but I
> was
> > > > wondering what the status of it is -- it seems to be lacking some
> > > features
> > > > that I thought it'd have (ex. reporting counters), and I'm not sure
> > > there's
> > > > an ability to extend the metrics being produced. Is this something
> that
> > > is
> > > > still being worked on, or is a legacy component? We are going to
> have to
> > > > write our own Service implementation, as the only supported one
> seems to
> > > be
> > > > Graphite, and wanted to make sure we're not going down a legacy path.
> > > >
> > > > Thanks!
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message