tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <rbalamo...@apache.org>
Subject Re: Tez Analyzer Usage Question
Date Tue, 18 Aug 2015 14:28:23 GMT
There is no rendering mechanism OOTB for displaying the job analyzer
results. Tool base jar which Bikas mentioned would definitely simplify the
way the analyzers are run and print the results on to console. Very basic
(PoC mode) standalone UI patch which uses Vaadin web framework is uploaded
in TEZ-2729. Please note that this is very very basic implementation which
lets user to run through a bunch of analyzers and render the results in
separate tabs.

Alternatively, tools like Zeppelin can be used for displaying the results
of job-analyzers. Advantage is that, it allows data to be stored in RDD
which can be linked to DataFrame and queried at runtime. It supports basic
charting as well. E.g notebook implementation is provided in
https://github.com/rajeshbalamohan/zeppelin-jobAnalyzer-demo-ATSData
(needless to say, this notebook is like PoC mode. :)  ).

~Rajesh.B


On Tue, Aug 18, 2015 at 3:48 AM, Jonathan Eagles <jeagles@gmail.com> wrote:

> Looking forward to it. Do you see this being tied to a tez cli as in
> TEZ-2658 <https://issues.apache.org/jira/browse/TEZ-2658> or as a stand
> alone?
> Jon
> <https://issues.apache.org/jira/browse/TEZ-2658>
>
> On Mon, Aug 17, 2015 at 5:12 PM, Bikas Saha <bikas@hortonworks.com> wrote:
>
> > I believe so :) with the eager developer caveats. I am working on a Tool
> > based jar so that the analysers could be run from the command line using
> > "Hadoop jar" instead of having to write code to invoke them. Please give
> me
> > a day or two for that.
> >
> > Bikas
> >
> > -----Original Message-----
> > From: Jonathan Eagles [mailto:jeagles@gmail.com]
> > Sent: Monday, August 17, 2015 3:01 PM
> > To: dev@tez.apache.org
> > Subject: Tez Analyzer Usage Question
> >
> > Can the job analyzer be used at this point for the eager developer? I
> > would like to get started using the functionality that is there. I have
> ATS
> > data for jobs and would be able to analyzer slow jobs, but don't have
> > instructions on what can be done at this point.
> >
> > I'm assuming there is a way to do this by downloading the ATS data,
> > parsing the ats data, and pointing the current set of analyzers at it.
> Just
> > needing a push in the right direction I think.
> >
> > Jon
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message