flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4490) Decouple Slot and Instance
Date Wed, 31 Aug 2016 17:18:21 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452798#comment-15452798
] 

ASF GitHub Bot commented on FLINK-4490:
---------------------------------------

GitHub user StephanEwen opened a pull request:

    https://github.com/apache/flink/pull/2447

    [FLINK-4490] [distributed coordination] Decouple the JobManage's slots from the 'Instance'

    To allow for a future dynamic slot allocation and release model, the slots should not
depend on 'Instance'.
    
    In this change, the Slots hold most of the necessary information directly (location, gateway)
and
    the interact with the Instance only via a 'SlotOwner' interface.
    
    The `ResourceID`, which has been introduces a while back, is a unique identifier for a
TaskManager that is  also constant for the lifetime of that particular TaskManager process.
Where previously, the JobManager comparted `Instance` objects for referential equality to
determine if two slots refer to the same TaskManager, the code now always refers to the `ResourceID`
instread.
    
    *Side note: I suggest to promote the `ResourceID` to `TaskManagerID` at some point.
    
    This also reworks the `InstanceConnectionInfo`to the `TaskManagerLocation`, which contains
in addition to IP/hostname/port also the `ResourceID`.
    
    All logic of the Scheduler and ExecutionGraph remains unchanged.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink slots_works

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2447.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2447
    
----
commit 198028f4ff1cc01e0e39bd389937a23b9ea45c7f
Author: Stephan Ewen <sewen@apache.org>
Date:   2016-08-29T14:58:31Z

    [FLINK-4490] [distributed coordination] (part 1) Change InstanceConnectionInfo to TaskManagerLocation
    
    This adds the ResourceId to the TaskManagerLocation

commit 02ad27e065e3f32b4503fac6ac5a3138a3b607e1
Author: Stephan Ewen <sewen@apache.org>
Date:   2016-08-30T18:34:20Z

    [FLINK-4490] [distributed coordination] (part 2) Make slots independent of 'Instance'.
    
    To allow for a future dynamic slot allocation and release model, the slots should not
depend on 'Instance'.
    In this change, the Slots hold most of the necessary information directly (location, gateway)
and
    the interact with the Instance only via a 'SlotOwner' interface.

commit e7efae594015e2788f840b0fc3c685880fd35e85
Author: Stephan Ewen <sewen@apache.org>
Date:   2016-08-31T11:52:45Z

    [FLINK-4525] [core] (followup) Remove remaining redundant code for pre-defined strictly
local assignments.

commit 7b30d8f6246af60b7fceebb0cdde761aee89a714
Author: Stephan Ewen <sewen@apache.org>
Date:   2016-08-31T11:59:01Z

    [FLINK-4490] [distributed coordination] (part 3) Rename methods on 'Instance' to have
more intuitive names
    
    getResourceID() --> getTaskManagerID()
    getInstanceConnectionInfo() --> getTaskManagerLocation()

----


> Decouple Slot and Instance
> --------------------------
>
>                 Key: FLINK-4490
>                 URL: https://issues.apache.org/jira/browse/FLINK-4490
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Distributed Coordination
>            Reporter: Kurt Young
>             Fix For: 1.2.0
>
>
> Currently, {{Slot}} and {{Instance}} holds each other. For {{Instance}} holding {{Slot}},
it makes sense because it reflects how many resources it can provide and how many are using.

> But it's not very necessary for {{Slot}} to hold {{Instance}} which it belongs to. It
only needs to hold some connection information and gateway to talk to. Another downside for
{{Slot}} holding {{Instance}} is that {{Instance}} actually contains some allocate/de-allocation
logicals, it will be difficult if we want to do some allocation refactor without letting {{Slot}}
noticed. 
> We should abstract the connection information of {{Instance}} to let {{Slot}} holds.
(Actually we have {{InstanceConnectionInfo}} now, but lacks of instance's akka gateway, maybe
we can just adding the akka gateway to the {{InstanceConnectionInfo}})



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message