hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] ashangit opened a new pull request #766: YARN-9509: Added a configuration for admins to be able to capped per-container cpu usage based on a multiplier
Date Wed, 24 Apr 2019 13:03:16 GMT
ashangit opened a new pull request #766: YARN-9509: Added a configuration for admins to be
able to capped per-container cpu usage based on a multiplier
URL: https://github.com/apache/hadoop/pull/766
 
 
   Add a multiplier configuration on strict resource usage to authorize container to use spare
cpu up to a limit.
   Currently with strict resource usage you can't get more than what you request which is
sometime not good for jobs that doesn't have a constant usage of cpu (for ex. spark jobs with
multiple stages).
   But without strict resource usage we have seen some bad behaviour from our users that don't
tune at all their needs and it leads to some containers requesting 2 vcore but constantly
using 20.
   The idea here is to still authorize containers to get more cpu than what they request if
some are free but also to avoid too big differencies so SLA on jobs is not breached if the
cluster is full (at least increase of runtime is contain)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message