samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xinyuiscool <...@git.apache.org>
Subject [GitHub] samza pull request #127: SAMZA-1204: Visualize StreamGraph and ExecutionPlan
Date Tue, 18 Apr 2017 21:52:32 GMT
GitHub user xinyuiscool opened a pull request:

    https://github.com/apache/samza/pull/127

    SAMZA-1204: Visualize StreamGraph and ExecutionPlan

    First look: https://xinyuiscool.github.io/visualizer/plan.html. This is based on the example
graph JSON generated in TestJobGraphJsonGenerator.
    
    For the api-test job:
    ![image](https://cloud.githubusercontent.com/assets/24738423/25154090/00f6cd76-2444-11e7-84aa-efa7d5fb735f.png)
    
    For the join-test job:
    ![image](https://cloud.githubusercontent.com/assets/24738423/25154115/19de1182-2444-11e7-9283-3b68a122201d.png)
    
    How to visualize the plan (locally):
    Once a Samza application (using fluent API) is deployed, an execution plan will be generated
by the ExecutionPlanner. The plan JSON will be written to a file (plan.json) under the bin
directory, which also contains the plan.html and javscripts (js folder). So the bin dir has
the following for visualization:
    ./bin
      - plan.json
      - plan.html
      - /js
        - d3.js
        - dagre.js
        - planToDagre.js
    
    Then deployed locally, the user can directly view the plan by enter the following in a
browser:
      **file:///my/samza/app/dir/bin/plan.html**
    
    To visualize the plan remotely, the user can either scp the files into local box and view
the plan there, or bring up a simple http server to serve the page.
    
    In the future when we have the Samza dashboard, we can add the plan visualization there
to avoid the above manual steps.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/xinyuiscool/samza SAMZA-1204

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/127.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #127
    
----
commit 0079c6d78e22de82c2857b20d8ae5e7436d38efa
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-10T23:53:19Z

    SAMZA-1204: Visualize StreamGraph and ExecutionPlan

commit beaf1bde2dd2acbf891639ffd1d06a69e38ac19d
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-11T16:04:38Z

    Minor adjustment to html

commit 40cf47d2cf08092acd4ddc540672ffce06768644
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-12T17:22:14Z

    Add support to operator call trace

commit d78be41bd243c6fc5cafd95aaea289eac34b511f
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-15T01:12:32Z

    Generate plan jon in the application runner

commit 16e3a4c6ee9d440418a6f40faefc1bbcafe9b893
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T19:03:28Z

    Further enhancement of the visualizer

commit 6eca9f85947e25bd40617eb97a0bd87ae42d77fe
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T19:12:28Z

    Merge branch 'master' into SAMZA-1204

commit 95d89211704e264533de6e0e9463f8f6d1bdc417
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T19:46:40Z

    Add application name and id

commit d3c2d4249b3948bfaf12828cbdb9b3a47b609157
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T21:47:54Z

    Update page title

commit 5e40fe5d3e17a5138b6ae7bcd64eeb28b1717fb5
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T23:00:11Z

    Fetch partition info for json

commit 7166d690856451d5161d430c5347726953aa4f2a
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T23:26:36Z

    Remove sample json

commit 7f447391e2b6a0eca04ac3990ad14dc015250a4f
Author: xinyuiscool <xinyuliu.us@gmail.com>
Date:   2017-04-17T23:32:25Z

    Use minified dagre

commit de3d43962306f5a213a83884c6d604cde821ada8
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-18T17:24:02Z

    Clean up the scripts

commit 1459ba93a509441de3d8edde32ec4e16b9be3df5
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-18T17:24:57Z

    Merge branch 'SAMZA-1204' of https://github.com/xinyuiscool/samza into SAMZA-1204

commit 4039ba2f9f36faa004b930887a804f7907b63594
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-18T19:14:44Z

    Clean up based on gradle build

commit 54325ab7fa9484ad442b68a7e32a59358f2beb83
Author: Xinyu Liu <xiliu@xiliu-ld.linkedin.biz>
Date:   2017-04-18T19:22:37Z

    Small javadoc change

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message