tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Eagles <jeag...@gmail.com>
Subject Re: Upload downloaded dag timeline data
Date Thu, 15 Oct 2015 16:53:37 GMT
Thanks Gopal for pointing this out. Thanks Sreenath for developing this. It
works like a charm.

Here are the exact steps I followed to get this to work. Posting so other
can follow if need be

*Development*

Viewing the downloaded DAG timeline data locally.

The mock-ats development tool has been generated that allows uploading the
DAG timeline data to a mock timeline server.



# build and run a stand alone timeline server

# pre-requisite node installed

brew install node

# get mock-ats an external tool

git clone https://github.com/sreenaths/mock-ats

cd mock-ats

# uploader uploads into mock-ats/data in a certain format. You may need to
clear mock-ats/data manually

# install on the node webserver dependencies

npm install

# launch node server on port 8188

node server.js http://localhost 8188

# node server is running both an uploader and timeline server at port 8188

# navigate to http://127.0.0.1:8188/dagupload

# upload the dag timeline zip file

# keep the server running during UI testing below



# build and run the tez UI

mvn clean install -DskipTests -Dmaven.javadoc.skip -pl tez-ui -am

cd tez-ui/target/tez-ui-*-SNAPSHOT/

# launch the Tez UI on port 8000

python -m SimpleHTTPServer 8000

# navigate to

http://127.0.0.1:8000/

# the dag now can be debugged locally


jeagles

On Wed, Oct 14, 2015 at 1:18 PM, Gopal Vijayaraghavan <gopalv@apache.org>
wrote:

>
> Sreenath has a mock-ATS which accepts those .zip files.
>
> https://github.com/sreenaths/mock-ats
>
>
> Start that up and go to http://<host-name>:8188/dagupload to upload the
> zip files.
>
> Caveat: unlike LevelDB it doesn't do de-dups, so uploading the same DAG
> twice messes it up.
>
> Cheers,
> Gopal
>
> On 10/14/15, 11:11 AM, "Jonathan Eagles" <jeagles@gmail.com> wrote:
>
> >Has anyone had any experience uploading data that was downloaded from the
> >Tez UI? I am trying to debug some UI issues that certain jobs experience I
> >want to run on the same dag timeline data as well as be able to upload
> >issues filed in the community into my local timeline store. Likely, I'll
> >need to write a tool if there isn't any available.
> >
> >Jon
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message