spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <>
Subject [jira] [Commented] (SPARK-19782) Spark query available cores from application
Date Wed, 01 Mar 2017 11:35:51 GMT


Sean Owen commented on SPARK-19782:

I'm not clear what this means? Spark apps are configured to request a certain amount of resource
from the resource manager. They can already tailor their usage with dynamic allocation. The
resource manager is the thing in charge of how much resource exists and how much to give to
an app.

> Spark query available cores from application
> --------------------------------------------
>                 Key: SPARK-19782
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Spark Core
>    Affects Versions: 2.1.0
>            Reporter: Tom Lewis
> It might be helpful for Spark jobs to self regulate resources if they could query how
many cores exist on a executing system not just how many are being used at a given time.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message