flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7113) Make ClusterDescriptor independent of Flink cluster size
Date Thu, 06 Jul 2017 14:59:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076649#comment-16076649
] 

ASF GitHub Bot commented on FLINK-7113:
---------------------------------------

GitHub user tillrohrmann opened a pull request:

    https://github.com/apache/flink/pull/4271

    [FLINK-7113] Make ClusterDescriptor independent of cluster size

    This PR is based on #4270.
    
    In order to pull some of the cluster deployment logic out of the `AbstractYarnClusterDescriptor`,
this PR introduces a `ClusterSpecification`. This specification contains information about
the size of the cluster to be started. Instead of setting these values via setters directly
on the `ClusterDescriptor`, we provide the information at deployment time via the `ClusterSpecification`.
    
    The way I envision the `ClusterDescriptor` to work, is to only contain information which
is necessary to talk to a resource manager, such as Mesos or Yarn. All Flink cluster specific
information should be provided to the `deploy` methods. This also allows us better share code
between different `ClusterDescriptor`, e.g. Mesos and Yarn.
    
    This is only the start of a refactoring of the `ClusterDescriptor`. Much more information
can be pulled out of the `AbstractYarnClusterDescriptor`.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tillrohrmann/flink refactorClusterDescriptor

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/4271.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4271
    
----
commit 9cfb9718d7b64e253aaf636dc17b8939e2592a98
Author: Till Rohrmann <trohrmann@apache.org>
Date:   2017-07-06T07:33:18Z

    [FLINK-7110] [client] Add per-job cluster deployment interface

commit 0143d7e33b7d99ce8371e455458b730ba2c13d06
Author: Till Rohrmann <trohrmann@apache.org>
Date:   2017-07-06T12:00:21Z

    [FLINK-7113] Make ClusterDescriptor independent of cluster size
    
    The deploySession method now is given a ClusterSpecification which specifies the
    size of the cluster which it is supposed to deploy.

----


> Make ClusterDescriptor independent of Flink cluster size
> --------------------------------------------------------
>
>                 Key: FLINK-7113
>                 URL: https://issues.apache.org/jira/browse/FLINK-7113
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Cluster Management
>    Affects Versions: 1.4.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>
> The {{ClusterDescriptor}} needs to know the size of the Flink cluster it is supposed
to deploy. As a consequence we have the {{AbstractYarnClusterDescriptor}} which is configured
with this information via setters. I think it would be better to give the cluster size to
the {{ClusterDescriptor}} via the {{deploySession(ClusterSpecification)}} call. That way we
better decouple the deployment from the cluster configuration.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message