flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7113) Make ClusterDescriptor independent of Flink cluster size
Date Thu, 06 Jul 2017 14:59:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076649#comment-16076649

ASF GitHub Bot commented on FLINK-7113:

GitHub user tillrohrmann opened a pull request:


    [FLINK-7113] Make ClusterDescriptor independent of cluster size

    This PR is based on #4270.
    In order to pull some of the cluster deployment logic out of the `AbstractYarnClusterDescriptor`,
this PR introduces a `ClusterSpecification`. This specification contains information about
the size of the cluster to be started. Instead of setting these values via setters directly
on the `ClusterDescriptor`, we provide the information at deployment time via the `ClusterSpecification`.
    The way I envision the `ClusterDescriptor` to work, is to only contain information which
is necessary to talk to a resource manager, such as Mesos or Yarn. All Flink cluster specific
information should be provided to the `deploy` methods. This also allows us better share code
between different `ClusterDescriptor`, e.g. Mesos and Yarn.
    This is only the start of a refactoring of the `ClusterDescriptor`. Much more information
can be pulled out of the `AbstractYarnClusterDescriptor`.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tillrohrmann/flink refactorClusterDescriptor

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4271
commit 9cfb9718d7b64e253aaf636dc17b8939e2592a98
Author: Till Rohrmann <trohrmann@apache.org>
Date:   2017-07-06T07:33:18Z

    [FLINK-7110] [client] Add per-job cluster deployment interface

commit 0143d7e33b7d99ce8371e455458b730ba2c13d06
Author: Till Rohrmann <trohrmann@apache.org>
Date:   2017-07-06T12:00:21Z

    [FLINK-7113] Make ClusterDescriptor independent of cluster size
    The deploySession method now is given a ClusterSpecification which specifies the
    size of the cluster which it is supposed to deploy.


> Make ClusterDescriptor independent of Flink cluster size
> --------------------------------------------------------
>                 Key: FLINK-7113
>                 URL: https://issues.apache.org/jira/browse/FLINK-7113
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Cluster Management
>    Affects Versions: 1.4.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
> The {{ClusterDescriptor}} needs to know the size of the Flink cluster it is supposed
to deploy. As a consequence we have the {{AbstractYarnClusterDescriptor}} which is configured
with this information via setters. I think it would be better to give the cluster size to
the {{ClusterDescriptor}} via the {{deploySession(ClusterSpecification)}} call. That way we
better decouple the deployment from the cluster configuration.

This message was sent by Atlassian JIRA

View raw message