hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaoqiao He <hexiaoq...@apache.org>
Subject Re: [E] Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4
Date Sun, 03 Jan 2021 14:55:38 GMT
Hi folks, Thanks for your try and vote for RC4.

I just prepare RC5 following the comment/ack and call another release
vote[1].

To Masatake Iwasaki,

> The step #4 of "Branching" section in HowToRelease is relevant instruction.
> https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease
> | 4. Release branch (branch-X.Y.Z) updates:
> |    a. Update hadoop-project/src/site/markdown/index.md.vm to reflect the
> right versions, new features and big improvements.

index.md.vm has updated at RC5.

To Sunil,

> I am a bit confused after seeing both *.sha512 and *.md5 files in the RC
> directory.

RC5 only including *.sha512 and no md5 checksum including.

To Sunil, Jim and Eric,

> However, while I did the tests to verify the RC, I ran into a serious NPE
> in YARN.
> I raised YARN-10540 <https://issues.apache.org/jira/browse/YARN-10540> to
> analyze this further. I think this issue due to YARN-10450
> <https://issues.apache.org/jira/browse/YARN-10450>.

YARN-10540 was involved in 3.2.2-RC5 and it is resolved now.

Please have another try and give the feedback/vote for RC5.

Thanks All,
- He Xiaoqiao

[1]
https://lists.apache.org/thread.html/r6c2817770e99362da79ebfaa0b6a774c5fca0f31e0dcc24df398eda0%40%3Chdfs-dev.hadoop.apache.org%3E

On Thu, Dec 24, 2020 at 11:30 AM Xiaoqiao He <hexiaoqiao@apache.org> wrote:

> Thanks Sunil, it makes sense to me. Will backport and prepare another RC
> shortly.
>
> - He Xiaoqiao
>
> On Wed, Dec 23, 2020 at 10:59 PM Sunil Govindan <sunilg@apache.org> wrote:
>
>> Hi Xiaoqiao
>> This is not an issue. You are missing the CORS config in YARN. I have
>> updated those configurations in the jira.
>> I feel that we can call a new RC as YARN-10540 is backported already.
>>
>> Thanks
>> Sunil
>>
>> On Wed, Dec 23, 2020 at 3:54 PM Xiaoqiao He <hexiaoqiao@apache.org>
>> wrote:
>>
>> > Thanks Jim, Eric and Sunil,
>> >
>> > I tried to backport YARN-10540 to branch-3.2.2 at local and check yarn
>> > ui2, there is still one link that does not work correctly, But I am not
>> > sure if it is expected. Detailed information is attached at YARN-10540.
>> > If someone could confirm or update, please let me know. I would like to
>> > backport then and prepare another RC.
>> > NOTE: Hadoop-3.2.2 will not release as we plan because some expected
>> > issues are in progress. It will be postponed to 2021/01, and I will try
>> my
>> > best to push it forward. AND Thanks to all reviewers (too numerous to
>> > mention).
>> >
>> > - He Xiaoqiao
>> >
>> >
>> > On Tue, Dec 22, 2020 at 8:32 AM Wei-Chiu Chuang <weichiu@apache.org>
>> > wrote:
>> >
>> >> We should target
>> >> 3.4.0,
>> >> 3.3.1
>> >> 3.2.3 (or 3.2.2)
>> >>
>> >> I updated and set 3.3.0 released in YARN.
>> >>
>> >>
>> >> Time to clean up some of the old releases. I'll do that.
>> >> On Mon, Dec 21, 2020 at 4:04 PM Eric Badger
>> >> <ebadger@verizonmedia.com.invalid> wrote:
>> >>
>> >>> I've committed https://issues.apache.org/jira/browse/YARN-10540.
>> >>> Xiaoqiao,
>> >>> feel free to cherry-pick this into the 3.2.2 release branch if you
>> think
>> >>> it
>> >>> is relevant.
>> >>>
>> >>> Also, can someone tell me which releases we should be targeting?
>> >>> Currently
>> >>> these versions are all Unreleased on JIRA:
>> >>> 3.4.1, 3.4.0
>> >>> 3.3.1, 3.3.0
>> >>> 3.2.3, 3.2.2
>> >>>
>> >>> As far as I know, neither trunk nor 3.3 have a release going on. So
I
>> >>> don't
>> >>> know why there are 2 versions as unreleased there.
>> >>>
>> >>> Eric
>> >>>
>> >>> On Mon, Dec 21, 2020 at 3:20 PM Jim Brennan
>> >>> <james.brennan@verizonmedia.com.invalid> wrote:
>> >>>
>> >>> > I put up a patch for
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_YARN-2D10540&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=KVdP1SUmHYb-tZP8tcigmw&m=WEMl9mt9SZiq7P20DPgeZO69TFwf3d0eldQGRWlLyQg&s=FdkrkiWqHwuibT6qFqvApTkO5aTimLx7WeP74tka5XM&e=
>> >>> > .
>> >>> > Thanks for bringing it to my attention.
>> >>> > Jim
>> >>> >
>> >>> > On Mon, Dec 21, 2020 at 10:36 AM Sunil Govindan <sunilg@apache.org>
>> >>> wrote:
>> >>> >
>> >>> > > I had some offline talks with a few folks.
>> >>> > > This issue is happening only in Mac, hence ideally it does
not
>> cause
>> >>> much
>> >>> > > of a problem in the supported OS.
>> >>> > >
>> >>> > > I will wait for feedback here to see whether we need another
RC by
>> >>> fixing
>> >>> > > this. And will continue the discussion in the jira.
>> >>> > >
>> >>> > > Thanks
>> >>> > > Sunil
>> >>> > >
>> >>> > > On Sat, Dec 19, 2020 at 11:07 PM Sunil Govindan <
>> sunilg@apache.org>
>> >>> > wrote:
>> >>> > >
>> >>> > > > Thanks, Xiaoqiao.
>> >>> > > > All files are looking good.
>> >>> > > >
>> >>> > > > However, while I did the tests to verify the RC, I ran
into a
>> >>> serious
>> >>> > NPE
>> >>> > > > in YARN.
>> >>> > > > I raised YARN-10540 <
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_YARN-2D10540&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=8i-pN_j9VKNxmOzU6gYGtWm_IVyeZkBcMwVI2eyzpRk&e=
>> >>> > > > to
>> >>> > > > analyze this further. I think this issue due to YARN-10450
>> >>> > > > <
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_YARN-2D10450&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=QEHMGtEbBz5Gn7mW4UsGlc-wNZ8ugZwiFQBy2pTx-Fw&e=
>> >>> > > >.
>> >>> > > > In the trunk, I am not able to see this issue. So It
could be
>> >>> possible
>> >>> > > > that some patches are not backported to branch-3.2.2.
>> >>> > > >
>> >>> > > > UI1 & UI2 nodes page is not working at this moment.
I will
>> check a
>> >>> bit
>> >>> > > > more to see about this and update here.
>> >>> > > >
>> >>> > > > Thanks
>> >>> > > > Sunil
>> >>> > > >
>> >>> > > > On Sat, Dec 19, 2020 at 5:36 PM Xiaoqiao He <
>> hexiaoqiao@apache.org
>> >>> >
>> >>> > > wrote:
>> >>> > > >
>> >>> > > >> Thanks Sunil, md5 files have been removed from RC4.
Please
>> have a
>> >>> > look.
>> >>> > > >> Thanks & Regards.
>> >>> > > >>
>> >>> > > >> - He Xiaoqiao
>> >>> > > >>
>> >>> > > >> On Sat, Dec 19, 2020 at 7:22 PM Sunil Govindan <
>> sunilg@apache.org
>> >>> >
>> >>> > > wrote:
>> >>> > > >>
>> >>> > > >>> Hi Xiaoqiao,
>> >>> > > >>>
>> >>> > > >>> Please remove the md5 files from your shared
RC4 repo. Thanks,
>> >>> @Akira
>> >>> > > >>> Ajisaka <aajisaka@apache.org> for sharing
this input.
>> >>> > > >>>
>> >>> > > >>> Thanks
>> >>> > > >>> Sunil
>> >>> > > >>>
>> >>> > > >>> On Sat, Dec 19, 2020 at 10:21 AM Sunil Govindan
<
>> >>> sunilg@apache.org>
>> >>> > > >>> wrote:
>> >>> > > >>>
>> >>> > > >>>> Reference:
>> >>> > > >>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.apache.org_dev_release-2Ddistribution.html-23sigs-2Dand-2Dsums&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=0qrQgqFXZzLqTDPzH_T1emam7NnHvnzXqZ6Ag0ccgIQ&e=
>> >>> > > >>>> Also, we had a Jira to track this HADOOP-15930
>> >>> > > >>>> <
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_HADOOP-2D15930&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=afUJEs9zxobnuEnC4v0xJ6v7d0dtrUkUaX15lI6VZbM&e=
>> >>> > > >.
>> >>> > > >>>>
>> >>> > > >>>> Thanks
>> >>> > > >>>> Sunil
>> >>> > > >>>>
>> >>> > > >>>> On Sat, Dec 19, 2020 at 10:16 AM Sunil Govindan
<
>> >>> sunilg@apache.org>
>> >>> > > >>>> wrote:
>> >>> > > >>>>
>> >>> > > >>>>> Hi Xiaoqiao and Wei-chiu
>> >>> > > >>>>>
>> >>> > > >>>>> I am a bit confused after seeing both
*.sha512 and *.md5
>> files
>> >>> in
>> >>> > the
>> >>> > > >>>>> RC directory.
>> >>> > > >>>>> Are we releasing both now?
>> >>> > > >>>>>
>> >>> > > >>>>> Thanks
>> >>> > > >>>>> Sunil
>> >>> > > >>>>>
>> >>> > > >>>>> On Wed, Dec 9, 2020 at 10:32 PM Xiaoqiao
He <
>> >>> hexiaoqiao@apache.org
>> >>> > >
>> >>> > > >>>>> wrote:
>> >>> > > >>>>>
>> >>> > > >>>>>> Hi folks,
>> >>> > > >>>>>>
>> >>> > > >>>>>> The release candidate (RC4) for Hadoop-3.2.2
is available
>> now.
>> >>> > > >>>>>> There are 10 commits[1] differences
between RC4 and RC3[2].
>> >>> > > >>>>>>
>> >>> > > >>>>>> The RC4 is available at:
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__people.apache.org_-7Ehexiaoqiao_hadoop-2D3.2.2-2DRC4&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=HUI5h2vNLugUtOGnApsEXpk40-hQszUQyUNbfYRQeXU&e=
>> >>> > > >>>>>> The RC4 tag in github is here:
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_hadoop_tree_release-2D3.2.2-2DRC4&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=Z3pv4v__QPeuKK5Ol_DOqKD-3deFnGKSfXdhi0v_K1c&e=
>> >>> > > >>>>>> The maven artifacts are staged at:
>> >>> > > >>>>>>
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__repository.apache.org_content_repositories_orgapachehadoop-2D1296&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=XA6ozoshoAKllmFj5vMTlXjE-8Ee41jaKYiTxBmnWpA&e=
>> >>> > > >>>>>>
>> >>> > > >>>>>> You can find my public key at:
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__dist.apache.org_repos_dist_release_hadoop_common_KEYS&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=oJ2jBABd8kCf9bLKre7vOkc4T010inenJNexdbGlOdg&e=
>> >>> > > or
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__people.apache.org_keys_committer_hexiaoqiao.asc&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=r5d-JXE1QGikxkDXlH-K9P15VaTM_kOtQz0IyBHM_Yk&e=
>> >>> > > directly.
>> >>> > > >>>>>>
>> >>> > > >>>>>> Please try the release and vote.
>> >>> > > >>>>>>
>> >>> > > >>>>>> I have done a simple testing with
my pseudo cluster.
>> >>> > > >>>>>> * Verify gpg sign and md5sum.
>> >>> > > >>>>>> * Setup pseudo cluster with HDFS
and YARN.
>> >>> > > >>>>>> * Run simple FsShell - mkdir/put/get/rename/mv.
>> >>> > > >>>>>> * Submit example mr job and check
the result -
>> Pi/wordcount.
>> >>> > > >>>>>> * Check the web UI of
>> >>> > NameNode/DataNode/Resourcemanager/NodeManager.
>> >>> > > >>>>>> My +1 to start.
>> >>> > > >>>>>>
>> >>> > > >>>>>> Thanks,
>> >>> > > >>>>>> He Xiaoqiao
>> >>> > > >>>>>>
>> >>> > > >>>>>> [1]
>> >>> > > >>>>>>
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_hadoop_compare_release-2D3.2.2-2DRC3...release-2D3.2.2-2DRC4&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=uUm4bp4BGiN0L04RJ1ny8G0ugpng57O_0Im8zQfMMGc&e=
>> >>> > > >>>>>> [2]
>> >>> > > >>>>>>
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.apache.org_thread.html_rfb74c3a5d4f223c5804d8ee622829263740cd8701c8f3fc8b6f970af-2540-253Chdfs-2Ddev.hadoop.apache.org-253E&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=x0gXxcI_Gd1cGJRczQYxrTznWYSJcZqCkUHuUPUjL7w&e=
>> >>> > > >>>>>> [3]
>> >>> > > >>>>>>
>> >>> > > >>>>>>
>> >>> > >
>> >>> >
>> >>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_secure_Dashboard.jspa-3FselectPageId-3D12335948&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw&m=nSlLXPsCxZGl0VV03dBWreCNrSH0SsNAZzmjRWO-2Zg&s=9w6gMkXXqF9gv5R3eFWF9fWH5mAJpqw7xXpGKd-XwfU&e=
>> >>> > > >>>>>>
>> >>> > > >>>>>
>> >>> > >
>> >>> >
>> >>>
>> >>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message