hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Brennan <james.bren...@verizonmedia.com.INVALID>
Subject Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2
Date Wed, 21 Aug 2019 22:11:57 GMT
Thanks for following up on the rolling upgrades!
We are definitely interested in seeing this move forward.

On Wed, Aug 21, 2019 at 2:37 PM Jonathan Hung <jyhung2357@gmail.com> wrote:

> Reviving this thread: we tested YARN RU starting with a cluster running
> 2.7.4, to running branch-2 + YARN-8200. Ran some simple MR/Spark jobs
> concurrently with the RM/NM upgrades and did not see any issues.
>
> If no other concerns I'll continue with a vote.
>
> Jonathan Hung
>
>
> On Thu, Apr 18, 2019 at 5:12 PM Jonathan Hung <jyhung2357@gmail.com>
> wrote:
>
>> Sorry for the delay, had to deprioritize this. Hoping to get to this next
>> week.
>>
>> Jonathan
>>
>> ------------------------------
>> *From:* Jim Brennan <james.brennan@verizonmedia.com>
>> *Sent:* Thursday, April 18, 2019 7:28 AM
>> *To:* Jonathan Hung
>> *Cc:* yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> *Subject:* Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2
>>
>> Hi Jonathan,
>>
>> Hi Jim, we have not tested rolling upgrade. I don’t foresee this being an
>>> issue, but we’ll try it out and report back.
>>
>>
>> Any update on this?
>> Jim
>>
>>
>> On Wed, Apr 3, 2019 at 2:16 AM Jonathan Hung <jyhung2357@gmail.com>
>> wrote:
>>
>>> Hi Jim, we have not tested rolling upgrade. I don’t foresee this being
>>> an issue, but we’ll try it out and report back.
>>>
>>> Jonathan
>>>
>>> ------------------------------
>>> *From:* Jim Brennan <james.brennan@verizonmedia.com>
>>> *Sent:* Tuesday, April 2, 2019 9:17 AM
>>> *To:* Jonathan Hung
>>> *Cc:* yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> *Subject:* Re: [DISCUSS] Merging YARN-8200 to branch-3.0 and branch-2
>>>
>>> Thanks for working on this!
>>> One concern for us is support for a rolling upgrade.  If we are running
>>> a cluster based on branch-2.8, will we be able to do a rolling upgrade (no
>>> cluster down-time) to a branch containing these changes?  Have you tested
>>> rolling upgrades?
>>>
>>> Thanks.
>>> Jim
>>>
>>> On Fri, Mar 29, 2019 at 2:14 PM Jonathan Hung <jyhung2357@gmail.com>
>>> wrote:
>>>
>>>> Hello devs,
>>>>
>>>> Starting a discuss thread to merge resource types/native GPU scheduling
>>>> support to branch-3.0 and branch-2. The resource types work was done in
>>>> trunk~branch-3.0 and GPU support done in trunk~branch-3.1, so the
>>>> proposal
>>>> is to merge GPU support into branch-3.0 and both resource types/GPU
>>>> support
>>>> to branch-2.
>>>>
>>>> Internally we've been running resource types/GPU support off a fork of
>>>> branch-2.9.0 in a > 300 node GPU cluster for a few months which has
>>>> worked
>>>> well. Also for completeness we verified that everything going into
>>>> branch-2
>>>> also exists in branch-3.0.
>>>>
>>>> The specific list of patches to merge is in feature branch
>>>> YARN-8200.branch3 (for branch-3.0) and feature branch YARN-8200 (for
>>>> branch-2). Full patches containing the YARN-8200.branch3 -> branch-3.0
>>>> diff
>>>> and YARN-8200 -> branch-2 diff have been posted to YARN-8200 jira.
>>>>
>>>> If there's no issues from the community I'll start a merge vote next
>>>> week.
>>>> Thanks.
>>>>
>>>> Jonathan Hung
>>>>
>>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message