hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hung <jyhung2...@gmail.com>
Subject Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2
Date Wed, 21 Aug 2019 19:36:52 GMT
Reviving this thread: we tested YARN RU starting with a cluster running
2.7.4, to running branch-2 + YARN-8200. Ran some simple MR/Spark jobs
concurrently with the RM/NM upgrades and did not see any issues.

If no other concerns I'll continue with a vote.

Jonathan Hung


On Thu, Apr 18, 2019 at 5:12 PM Jonathan Hung <jyhung2357@gmail.com> wrote:

> Sorry for the delay, had to deprioritize this. Hoping to get to this next
> week.
>
> Jonathan
>
> ------------------------------
> *From:* Jim Brennan <james.brennan@verizonmedia.com>
> *Sent:* Thursday, April 18, 2019 7:28 AM
> *To:* Jonathan Hung
> *Cc:* yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> *Subject:* Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2
>
> Hi Jonathan,
>
> Hi Jim, we have not tested rolling upgrade. I don’t foresee this being an
>> issue, but we’ll try it out and report back.
>
>
> Any update on this?
> Jim
>
>
> On Wed, Apr 3, 2019 at 2:16 AM Jonathan Hung <jyhung2357@gmail.com> wrote:
>
>> Hi Jim, we have not tested rolling upgrade. I don’t foresee this being an
>> issue, but we’ll try it out and report back.
>>
>> Jonathan
>>
>> ------------------------------
>> *From:* Jim Brennan <james.brennan@verizonmedia.com>
>> *Sent:* Tuesday, April 2, 2019 9:17 AM
>> *To:* Jonathan Hung
>> *Cc:* yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> *Subject:* Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2
>>
>> Thanks for working on this!
>> One concern for us is support for a rolling upgrade.  If we are running a
>> cluster based on branch-2.8, will we be able to do a rolling upgrade (no
>> cluster down-time) to a branch containing these changes?  Have you tested
>> rolling upgrades?
>>
>> Thanks.
>> Jim
>>
>> On Fri, Mar 29, 2019 at 2:14 PM Jonathan Hung <jyhung2357@gmail.com>
>> wrote:
>>
>>> Hello devs,
>>>
>>> Starting a discuss thread to merge resource types/native GPU scheduling
>>> support to branch-3.0 and branch-2. The resource types work was done in
>>> trunk~branch-3.0 and GPU support done in trunk~branch-3.1, so the
>>> proposal
>>> is to merge GPU support into branch-3.0 and both resource types/GPU
>>> support
>>> to branch-2.
>>>
>>> Internally we've been running resource types/GPU support off a fork of
>>> branch-2.9.0 in a > 300 node GPU cluster for a few months which has
>>> worked
>>> well. Also for completeness we verified that everything going into
>>> branch-2
>>> also exists in branch-3.0.
>>>
>>> The specific list of patches to merge is in feature branch
>>> YARN-8200.branch3 (for branch-3.0) and feature branch YARN-8200 (for
>>> branch-2). Full patches containing the YARN-8200.branch3 -> branch-3.0
>>> diff
>>> and YARN-8200 -> branch-2 diff have been posted to YARN-8200 jira.
>>>
>>> If there's no issues from the community I'll start a merge vote next
>>> week.
>>> Thanks.
>>>
>>> Jonathan Hung
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message