samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srujun Thanmay Gupta <sgupt...@illinois.edu>
Subject Re: No Metrics Emitted in new High-Level API
Date Tue, 30 Jan 2018 22:38:33 GMT
Hi,

Thank you Yi, what you mentioned seemed to be the issue. Configuring the
metrics serdes fixed the metrics output. The reason I had not added them
initially was because the wikipedia-application.properties file in the
hello-samza example here: (
https://github.com/apache/samza-hello-samza/blob/master/src/main/config/wikipedia-application.properties
)
does not have them configured.

Using that configuration for the wikipedia-application demo did not produce
metrics. But, surprisingly, not configuring the serdes for the
wikipedia-feed demo either (which uses the low-level API) worked fine,
which is why I was wondering if metrics is configured differently when
using the high-level API. I've linked the two demos below, with the
.properties files used for each and the DEBUG logs generated.

https://github.com/srujun/srujun.github.io/tree/master/files/samza/wikipedia-feed
(metrics works as expected, emitting JSON-encoded messages to kafka.metrics)
https://github.com/srujun/srujun.github.io/tree/master/files/samza/wikipedia-application
(metrics not working unless serdes are also configured)

>From what I can see, there are metrics serialization errors in the
wikipedia-application logs, as expected. But no such issue for
wikipedia-feed, even though no serde was configured. Is there some other
issue here?

Thank you again.

On Fri, Jan 26, 2018 at 11:00 PM, Yi Pan <nickpan47@gmail.com> wrote:

> Hi, Srujun,
>
> I noticed that you did not configure the metrics serde as stated here:
> http://samza.apache.org/learn/documentation/0.14/container/metrics.html.
> Could that be the reason?
>
> -Yi
>
> On Fri, Jan 26, 2018 at 7:58 PM, Jagadish Venkatraman <
> jagadish1989@gmail.com> wrote:
>
> > Can you turn on DEBUG logging for the MetricsSnapshotReporter class and
> > re-attach the logs? I cannot observe any obvious red-flag from your logs.
> >
> > On Fri, Jan 26, 2018 at 7:13 PM, Srujun Thanmay Gupta <
> > sgupta80@illinois.edu
> > > wrote:
> >
> > > Hello Yi,
> > >
> > > The logs are less than 100KB each, I'm not sure if that's too large for
> > the
> > > mailing list system. I've uploaded the log files to the links below, in
> > > case others would like to look at them too.
> > >
> > > http://srujun.com/files/samza/wikipedia-application_samza-
> > container-0.log
> > > http://srujun.com/files/samza/wikipedia-application_samza-
> > > job-coordinator.log
> > > http://srujun.com/files/samza/wikipedia-feed_samza-container-0.log
> > > http://srujun.com/files/samza/wikipedia-feed_samza-job-coordinator.log
> > >
> > > Thank you for your help!
> > >
> > > --
> > > Srujun Thanmay Gupta
> > >
> >
> >
> >
> > --
> > Jagadish V,
> > Graduate Student,
> > Department of Computer Science,
> > Stanford University
> >
>



-- 
Srujun Thanmay Gupta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message