flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8431) Allow to specify # GPUs for TaskManager in Mesos
Date Mon, 29 Jan 2018 19:23:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343854#comment-16343854

ASF GitHub Bot commented on FLINK-8431:

Github user EronWright commented on the issue:

    @tillrohrmann regarding your general question, you are right that Flink could, in concept,
deploy to GPU hosts even if Flink doesn't require any GPUs.   But we should keep in mind the
intent of `GPU_RESOURCES`, which is to reserve GPU hosts for frameworks that need GPUs with
an opt-in mechanism.     
    We could adjust the logic in this PR to the effect that the `gpus` configuration option
has no default value.  If any value is configured (including `0.0`), add the `GPU_RESOURCES`.
 This would allow the user to make use of GPU hosts even if not requiring any GPU resources,
but it feels a bit exploitative and would prefer we not do this.  WDYT?

> Allow to specify # GPUs for TaskManager in Mesos
> ------------------------------------------------
>                 Key: FLINK-8431
>                 URL: https://issues.apache.org/jira/browse/FLINK-8431
>             Project: Flink
>          Issue Type: Improvement
>          Components: Cluster Management, Mesos
>            Reporter: Dongwon Kim
>            Assignee: Dongwon Kim
>            Priority: Minor
> Mesos provides first-class support for Nvidia GPUs [1], but Flink does not exploit it
when scheduling TaskManagers. If Mesos agents are configured to isolate GPUs as shown in [2],
TaskManagers that do not specify to use GPUs cannot see GPUs at all.
> We, therefore, need to introduce a new configuration property named "mesos.resourcemanager.tasks.gpus"
to allow users to specify # of GPUs for each TaskManager process in Mesos.
> [1] http://mesos.apache.org/documentation/latest/gpu-support/
> [2] http://mesos.apache.org/documentation/latest/gpu-support/#agent-flags

This message was sent by Atlassian JIRA

View raw message