hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wangda Tan <wheele...@gmail.com>
Subject Re: [VOTE] Merge Resource Types (YARN-3926) to branch-3.0
Date Sat, 28 Oct 2017 03:43:04 GMT
+1 (Binding), Thanks Daniel for pushing this!

Regarding to performance:
- Tested with a 8K nodes SLS trace (2 resource types), doesn't see any
noticeable regression comparing to trunk.
- Tested with TestCapacitySchedulerPerf, doesn't see any noticeable
regression as well.

Regarding to end-to-end test:
- Configured a pseudo cluster with 4 resource types, run jobs (After
applied YARN-6927). Everything looks fine.

Thanks,
Wangda


On Tue, Oct 24, 2017 at 11:08 AM, Andrew Wang <andrew.wang@cloudera.com>
wrote:

> +0
>
> On Tue, Oct 24, 2017 at 10:56 AM, Daniel Templeton <daniel@cloudera.com>
> wrote:
>
> > I'd like to formally start the voting process for merging the
> > resource-types branch into branch-3.0.  The resource-types branch is a
> > selective backport of JIRAs that were already merged into trunk in a
> > previous merge vote for YARN-3926 (resource types) [1].  For a full
> > explanation of the feature, benefits, and risks, see the previous DISCUSS
> > thread [2].  The vote will be 7 days, ending Tuesday Oct 31 at 11:00AM
> PDT.
> >
> > In summary, resource types adds the ability to declaratively configure
> new
> > resource types in addition to CPU and memory and request them when
> > submitting resource requests.  The resource-types branch currently
> > represents 32 patches from trunk drawn from the resource types umbrella
> > JIRAs: YARN-3926 [3] and YARN-7069 [4].
> >
> > Key points:
> > * If no additional resource types are configured, the user experience
> with
> > YARN remains unchanged.
> > * Performance is the primary risk. We have been closely watching the
> > performance impact of adding resource types, and according to current
> > measurements the impact is trivial.
> > * This merge vote is for resource types excluding the resource profiles
> > feature which was included in the original merge vote [1].
> > * Documentation is available in trunk via YARN-7056 [5] with improvements
> > pending review in YARN-7369 [6].
> >
> > Refreshed performance numbers on the resource-types branch are pending,
> > and I'll post them to this thread as soon as they're ready.
> >
> > Thanks!
> > Daniel
> >
> > [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
> > 708.mbox/%3CCAD++eCm6xSs4_kXP4Audf85_rGg4pZxKuOx7u2VP8tfzmY4
> > Pcg@mail.gmail.com%3E
> > [2] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
> > 710.mbox/%3Caa2bcc6d-9d88-459d-63f4-5bb43e31f4f4%40cloudera.com%3E
> > [3] https://issues.apache.org/jira/browse/YARN-3926
> > [4] https://issues.apache.org/jira/browse/YARN-7069
> > [5] https://issues.apache.org/jira/browse/YARN-7056
> > [6] https://issues.apache.org/jira/browse/YARN-7369
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message