flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1295) Add option to Flink client to start a YARN session per job
Date Fri, 09 Jan 2015 16:22:35 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271472#comment-14271472
] 

ASF GitHub Bot commented on FLINK-1295:
---------------------------------------

GitHub user rmetzger opened a pull request:

    https://github.com/apache/flink/pull/292

    [FLINK-1295][FLINK-883] Allow to deploy 'job only' YARN cluster. Add tests to YARN

    - users can now also deploy Flink on YARN for executing a single job.
    - the flink-yarn project has been moved out of the flink-addons module
    - the MiniYARNCluster is used for testing Flink on YARN
    - There is now a (undocumented) Java interface Flink's YARN client, allowing users to
manually control the YARN session.
    - ALL ports used by Flink when running on YARN are automatically determined. In the past
users reported problems with blocked ports (YARN is telling the client the RPC address of
the application master)
    - The checks before deployment have been improved to give better error messages if the
user is requesting too many resources for a YARN session
    
    The change has been tested on google compute cloud ("click to deploy" with google cloud
storage, not hdfs) and amazon emr (hdfs).

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rmetzger/flink flink1295-flink883-rebased-after-akka

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/292.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #292
    
----
commit 273cafbf44b5796b96ff2396b0f6e68f79ce9185
Author: Robert Metzger <rmetzger@apache.org>
Date:   2014-12-01T17:59:49Z

    [FLINK-1295][FLINK-883] Allow to deploy 'job only' YARN cluster. Add tests to YARN
    
    - users can now also deploy Flink on YARN for executing a single job.
    - The flink-yarn project has been moved out of the flink-addons module
    - the MiniYARNCluster is used for testing Flink on YARN
    - There is now a (undocumented) Java interface Flink's YARN client, allowing users to
manually control the Yarn session.
    - ALL ports used by Flink when running on YARN are automatically determined. In the past
users reported problems with blocked ports (YARN is telling the client the RPC address of
the application master)
    - The checks before deployment have been improved to give better error messages if the
user is requesting too many resources for a YARN session

----


> Add option to Flink client to start a YARN session per job
> ----------------------------------------------------------
>
>                 Key: FLINK-1295
>                 URL: https://issues.apache.org/jira/browse/FLINK-1295
>             Project: Flink
>          Issue Type: New Feature
>          Components: YARN Client
>            Reporter: Robert Metzger
>            Assignee: Robert Metzger
>
> Currently, Flink users can only launch Flink on YARN as a "YARN session" (meaning a long-running
YARN application that can run multiple Flink jobs)
> Users have requested to extend the Flink Client to allocate YARN containers only for
executing a single job.
> As part of this pull request, I would suggest to refactor the YARN Client to make it
more modular and object oriented.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message