hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-9548) [Umbrella] Make YARN work well in elastic cloud environments
Date Mon, 13 May 2019 22:31:00 GMT
Vinod Kumar Vavilapalli created YARN-9548:
---------------------------------------------

             Summary: [Umbrella] Make YARN work well in elastic cloud environments
                 Key: YARN-9548
                 URL: https://issues.apache.org/jira/browse/YARN-9548
             Project: Hadoop YARN
          Issue Type: New Feature
            Reporter: Vinod Kumar Vavilapalli


YARN works well in static environments but there isn't fundamentally broken in YARN to stop
us from making it work well in dynamic environments like cloud (public or private) as well.

There are few areas where we need to invest though
 # Autoscaling
 -- cluster level: add/remove nodes intelligently based on metrics and/or admin plugins
 -- node level: scale nodes up/down vertically?
 # Smarter scheduling
    -- to pack containers as opposed to spreading them around to account for nodes going away
    -- to account for speculative nodes like spot instances
 # Handling nodes going away better
    -- by decommissioning sanely
    -- dealing with auxiliary services data
 # And any installation helpers in this dynamic world - scripts, operators etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message