flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5941) Let handlers take part in job archiving
Date Wed, 01 Mar 2017 17:04:45 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890555#comment-15890555

ASF GitHub Bot commented on FLINK-5941:

GitHub user zentol opened a pull request:


    [FLINK-5941] Integrate Archiver pattern into handlers

    This PR adds the last piece to the foundation of the HistoryServer. It builds on top of
#3365 and #3376, meaning that only 58649fb is relevant for this PR.
    The key addition is the ```Archiver``` interface. An ```Archiver``` generates Json responses
and their respective REST paths for a given ```AccessExecutionGraph```.
    The goal here is to allow writing these responses to files contained in a directory structure
resembling the REST API, without defining the structure in 2 different places. With the additions
made here the writing of files can be done in a simple loop:
    AccessExecutiongraph graph = ...
    for (Archiver archiver : WebRuntimeMonitor.getArchivers()) {
    	for (ArchivedJson archive : archiver.archive(graph)) {
    		File f = createFile(basePath, archive.path);
    		writeJson(f, archive.json);
    Instead of extending existing job-related handlers I've added a new class in each of them
that implements the new interface. This makes testing easier and also allows for easier refactoring/relocation.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 5941_handler_archiver

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3444
commit 61688e45a461da717c220891b60f3329f202ad7a
Author: zentol <chesnay@apache.org>
Date:   2017-03-01T11:23:15Z

    [FLINK-5870] Handlers define REST URLs

commit 8eb29ab3b6ef87fe9a1b00ad5ac08b6e7725c2f4
Author: zentol <chesnay@apache.org>
Date:   2017-02-28T17:01:14Z

    [FLINK-5852] Move handler JSON generation code into static methods

commit 58649fb99370b896c0c7bdcc81b674c7b7a81c56
Author: zentol <chesnay@apache.org>
Date:   2017-02-20T15:30:35Z

    [FLINK-5941] Integrate Archiver pattern into handlers


> Let handlers take part in job archiving
> ---------------------------------------
>                 Key: FLINK-5941
>                 URL: https://issues.apache.org/jira/browse/FLINK-5941
>             Project: Flink
>          Issue Type: New Feature
>          Components: Webfrontend
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>             Fix For: 1.3.0
> The key idea behind the HistoryServer is to pre-compute all JSON responses which the
WebFrontend could request and store them as files in a directory structure resembling the
> For this require a mechanism to generate the responses and their corresponding REST URL.
> FLINK-5852 made it easier to re-use the JSON generation code, while FLINK-5870 made handlers
aware of the REST URLs that they are registered one.
> The aim of this JIRA is to extend job-related handlers, building on the above JIRAs,
enabling them to generate a number of (Path, Json) pairs for a given ExecutionGraph, containing
all responses that they could generate for the given graph and their respective REST URL..

This message was sent by Atlassian JIRA

View raw message