hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Antal <adam.an...@cloudera.com.INVALID>
Subject Re: [DISCUSS] Hadoop 3.3.0 Release include ARM binary
Date Thu, 18 Jun 2020 14:19:28 GMT
YARN-10314 is also merged. I don't see any blockers at this point.
(Actually I couldn't see any jiras
<https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%2Fs%22%20%3D%203.3.0%20ORDER%20BY%20priority%20DESC>
targeted for 3.3.0).

In the community sync yesterday we wanted to discuss the 3.3.0 release, but
nobody had information about it in the call. Could you share the latest on
the upcoming 3.3.0 release?

Thanks,
Adam

On Mon, Jun 15, 2020 at 9:17 AM Ayush Saxena <ayushtkn@gmail.com> wrote:

> YARN-10314 also seems to be a blocker.
>
> https://issues.apache.org/jira/browse/YARN-10314
>
> We should wait for that as well, should get concluded in a day or two.
>
> -Ayush
>
> > On 15-Jun-2020, at 7:21 AM, Sheng Liu <liusheng2048@gmail.com> wrote:
> >
> > The  HADOOP-17046 <https://issues.apache.org/jira/browse/HADOOP-17046>
> has
> > been merged :)
> >
> > Brahma Reddy Battula <brahma@apache.org> 于2020年6月4日周四 下午10:43写道:
> >
> >> Following blocker is pending for 3.3.0 release which is ready for
> review.
> >> Mostly we'll have RC soon.
> >> https://issues.apache.org/jira/browse/HADOOP-17046
> >>
> >> Protobuf dependency was unexpected .
> >>
> >>> On Mon, Jun 1, 2020 at 7:11 AM Sheng Liu <liusheng2048@gmail.com>
> wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> It looks like the 3.3.0 branch has been created for quite a while. Not
> >> sure
> >>> if there is remain block issue that need to be addressed before Hadoop
> >>> 3.3.0 release publishing, maybe we can bring up to here and move the
> >>> release forward ?
> >>>
> >>> Thank.
> >>>
> >>> Brahma Reddy Battula <brahma@apache.org> 于2020年3月25日周三
上午1:55写道:
> >>>
> >>>> thanks to all.
> >>>>
> >>>> will make this as optional..will update the wiki accordingly.
> >>>>
> >>>> On Wed, Mar 18, 2020 at 12:05 AM Vinayakumar B <
> >> vinayakumarb@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Making ARM artifact optional, makes the release process simpler
for
> >> RM
> >>>> and
> >>>>> unblocks release process (if there is unavailability of ARM
> >> resources).
> >>>>>
> >>>>> Still there are possible options to collaborate with RM ( as brahma
> >>>>> mentioned earlier) and provide ARM artifact may be before or after
> >>> vote.
> >>>>> If feasible RM can decide to add ARM artifact by collaborating with
> >>>> @Brahma
> >>>>> Reddy Battula <brahma@apache.org> or me to get the ARM artifact.
> >>>>>
> >>>>> -Vinay
> >>>>>
> >>>>> On Tue, Mar 17, 2020 at 11:39 PM Arpit Agarwal
> >>>>> <aagarwal@cloudera.com.invalid> wrote:
> >>>>>
> >>>>>> Thanks for the clarification Brahma. Can you update the proposal
to
> >>>> state
> >>>>>> that it is optional (it may help to put the proposal on cwiki)?
> >>>>>>
> >>>>>> Also if we go ahead then the RM documentation should be clear
this
> >> is
> >>>> an
> >>>>>> optional step.
> >>>>>>
> >>>>>>
> >>>>>>> On Mar 17, 2020, at 11:06 AM, Brahma Reddy Battula <
> >>>> brahma@apache.org>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Sure, we can't make mandatory while voting and we can upload
to
> >>>>> downloads
> >>>>>>> once release vote is passed.
> >>>>>>>
> >>>>>>> On Tue, 17 Mar 2020 at 11:24 PM, Arpit Agarwal
> >>>>>>> <aagarwal@cloudera.com.invalid> wrote:
> >>>>>>>
> >>>>>>>>> Sorry,didn't get you...do you mean, once release
voting is
> >>>>>>>>> processed and upload by RM..?
> >>>>>>>>
> >>>>>>>> Yes, that is what I meant. I don’t want us to make
more
> >> mandatory
> >>>> work
> >>>>>> for
> >>>>>>>> the release manager because the job is hard enough already.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Mar 17, 2020, at 10:46 AM, Brahma Reddy Battula
<
> >>>>> brahma@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Sorry,didn't get you...do you mean, once release
voting is
> >>>> processed
> >>>>>> and
> >>>>>>>>> upload by RM..?
> >>>>>>>>>
> >>>>>>>>> FYI. There is docker image for ARM also which support
all
> >> scripts
> >>>>>>>>> (createrelease, start-build-env.sh, etc ).
> >>>>>>>>>
> >>>>>>>>> https://issues.apache.org/jira/browse/HADOOP-16797
> >>>>>>>>>
> >>>>>>>>> On Tue, Mar 17, 2020 at 10:59 PM Arpit Agarwal
> >>>>>>>>> <aagarwal@cloudera.com.invalid> wrote:
> >>>>>>>>>
> >>>>>>>>>> Can ARM binaries be provided after the fact?
We cannot
> >> increase
> >>>> the
> >>>>>> RM’s
> >>>>>>>>>> burden by asking them to generate an extra set
of binaries.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Mar 17, 2020, at 10:23 AM, Brahma Reddy
Battula <
> >>>>>> brahma@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> + Dev mailing list.
> >>>>>>>>>>>
> >>>>>>>>>>> ---------- Forwarded message ---------
> >>>>>>>>>>> From: Brahma Reddy Battula <brahma@apache.org>
> >>>>>>>>>>> Date: Tue, Mar 17, 2020 at 10:31 PM
> >>>>>>>>>>> Subject: Re: [DISCUSS] Hadoop 3.3.0 Release
include ARM
> >> binary
> >>>>>>>>>>> To: junping_du <junping_du@apache.org>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> thanks junping for your reply.
> >>>>>>>>>>>
> >>>>>>>>>>> bq.      I think most of us in Hadoop community
doesn't want
> >> to
> >>>>> have
> >>>>>>>>>> biased
> >>>>>>>>>>> on ARM or any other platforms.
> >>>>>>>>>>>
> >>>>>>>>>>> Yes, release voting will be based on the
source
> >>> code.AFAIK,Binary
> >>>>> we
> >>>>>>>> are
> >>>>>>>>>>> providing for user to easy to download and
verify.
> >>>>>>>>>>>
> >>>>>>>>>>> bq.     The only thing I try to understand
is how much
> >>> complexity
> >>>>> get
> >>>>>>>>>>> involved for our RM work. Does that potentially
become a
> >>> blocker
> >>>>> for
> >>>>>>>>>> future
> >>>>>>>>>>> releases? And how we can get rid of this
risk.
> >>>>>>>>>>>
> >>>>>>>>>>> As I mentioned earlier, RM need to access
the ARM machine(it
> >>> will
> >>>>> be
> >>>>>>>>>>> donated and current qbt also using one ARM
machine) and build
> >>> tar
> >>>>>> using
> >>>>>>>>>> the
> >>>>>>>>>>> keys. As it can be common machine, RM can
delete his keys
> >> once
> >>>>>> release
> >>>>>>>>>>> approved.
> >>>>>>>>>>> Can be sorted out as I mentioned earlier.(For
accessing the
> >> ARM
> >>>>>>>> machine)
> >>>>>>>>>>>
> >>>>>>>>>>> bq.       If you can list the concrete work
that RM need to
> >> do
> >>>>> extra
> >>>>>>>> for
> >>>>>>>>>>> ARM release, that would help us to better
understand.
> >>>>>>>>>>>
> >>>>>>>>>>> I can write and update for future reference.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Mar 17, 2020 at 10:41 AM 俊平堵
<junping_du@apache.org>
> >>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Brahma,
> >>>>>>>>>>>>  I think most of us in Hadoop community
doesn't want to
> >> have
> >>>>> biased
> >>>>>>>>>> on
> >>>>>>>>>>>> ARM or any other platforms.
> >>>>>>>>>>>>  The only thing I try to understand
is how much complexity
> >>> get
> >>>>>>>>>>>> involved for our RM work. Does that
potentially become a
> >>> blocker
> >>>>> for
> >>>>>>>>>> future
> >>>>>>>>>>>> releases? And how we can get rid of
this risk.
> >>>>>>>>>>>>   If you can list the concrete work
that RM need to do
> >> extra
> >>>> for
> >>>>>> ARM
> >>>>>>>>>>>> release, that would help us to better
understand.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Junping
> >>>>>>>>>>>>
> >>>>>>>>>>>> Akira Ajisaka <aajisaka@apache.org>
于2020年3月13日周五
> >> 上午12:34写道:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> If you can provide ARM release for
future releases, I'm
> >> fine
> >>>> with
> >>>>>>>> that.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Akira
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Mar 12, 2020 at 9:41 PM
Brahma Reddy Battula <
> >>>>>>>>>> brahma@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> thanks Akira.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Currently only problem is dedicated
ARM for future
> >> RM.This i
> >>>>> want
> >>>>>> to
> >>>>>>>>>>>>> sort
> >>>>>>>>>>>>>> out like below,if you've some
other,please let me know.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> i) Single machine and share
cred to future RM ( as we can
> >>>> delete
> >>>>>>>> keys
> >>>>>>>>>>>>> once
> >>>>>>>>>>>>>> release is over).
> >>>>>>>>>>>>>> ii) Creating the jenkins project
( may be we need to
> >> discuss
> >>>> in
> >>>>>> the
> >>>>>>>>>>>>>> board..)
> >>>>>>>>>>>>>> iii) I can provide ARM release
for future releases.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 5:14
PM Akira Ajisaka <
> >>>>>> aajisaka@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Brahma,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think we cannot do any
of your proposed actions.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://www.apache.org/legal/release-policy.html#owned-controlled-hardware
> >>>>>>>>>>>>>>>> Strictly speaking, releases
must be verified on hardware
> >>>> owned
> >>>>>> and
> >>>>>>>>>>>>>>> controlled by the committer.
That means hardware the
> >>>> committer
> >>>>>> has
> >>>>>>>>>>>>>> physical
> >>>>>>>>>>>>>>> possession and control of
and exclusively full
> >>>>>>>>>>>>> administrative/superuser
> >>>>>>>>>>>>>>> access to. That's because
only such hardware is qualified
> >>> to
> >>>>>> hold a
> >>>>>>>>>>>>> PGP
> >>>>>>>>>>>>>>> private key, and the release
should be verified on the
> >>>> machine
> >>>>>> the
> >>>>>>>>>>>>>> private
> >>>>>>>>>>>>>>> key lives on or on a machine
as trusted as that.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>> https://www.apache.org/dev/release-distribution.html#sigs-and-sums
> >>>>>>>>>>>>>>>> Private keys MUST NOT
be stored on any ASF machine.
> >>>> Likewise,
> >>>>>>>>>>>>>> signatures
> >>>>>>>>>>>>>>> for releases MUST NOT be
created on ASF machines.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We need to have dedicated
physical ARM machines for each
> >>>>> release
> >>>>>>>>>>>>> manager,
> >>>>>>>>>>>>>>> and now it is not feasible.
> >>>>>>>>>>>>>>> If you provide an unofficial
ARM binary release in some
> >>>>>> repository,
> >>>>>>>>>>>>>> that's
> >>>>>>>>>>>>>>> okay.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Akira
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at
7:57 PM Brahma Reddy Battula <
> >>>>>>>>>>>>> brahma@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hello folks,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> As currently trunk will
support ARM based compilation
> >> and
> >>>>> qbt(1)
> >>>>>>>> is
> >>>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>> from several months
with quite stable, hence planning to
> >>>>> propose
> >>>>>>>> ARM
> >>>>>>>>>>>>>>>> binary
> >>>>>>>>>>>>>>>> this time.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ( Note : As we'll know
voting will be based on the
> >>> source,so
> >>>>>> this
> >>>>>>>>>>>>> will
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>> issue.)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> *Proposed Change:*
> >>>>>>>>>>>>>>>> Currently in downloads
we are keeping only x86
> >>> binary(2),Can
> >>>>> we
> >>>>>>>> keep
> >>>>>>>>>>>>> ARM
> >>>>>>>>>>>>>>>> binary also.?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> *Actions:*
> >>>>>>>>>>>>>>>> a) *Dedicated* *Machine*:
> >>>>>>>>>>>>>>>>    i) Dedicated ARM
machine will be donated which I
> >>>> confirmed
> >>>>>>>>>>>>>>>>    ii) Or can use jenkins
ARM machine itself which is
> >>>>> currently
> >>>>>>>>>>>>> used
> >>>>>>>>>>>>>>>> for ARM
> >>>>>>>>>>>>>>>> b) *Automate Release:*
How about having one release
> >>> project
> >>>> in
> >>>>>>>>>>>>>> jenkins..?
> >>>>>>>>>>>>>>>> So that future RM's
just trigger the jenkin project.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Please let me know your
thoughts on this.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 1.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-qbt-linux-ARM-trunk/
> >>>>>>>>>>>>>>>> 2.https://hadoop.apache.org/releases.html
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --Brahma Reddy Battula
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --Brahma Reddy Battula
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --Brahma Reddy Battula
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --Brahma Reddy Battula
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail:
> >> yarn-dev-unsubscribe@hadoop.apache.org
> >>>>>>>>>> For additional commands, e-mail:
> >>> yarn-dev-help@hadoop.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --Brahma Reddy Battula
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >>>>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>>
> >>>>
> >>>>
> >>>> --Brahma Reddy Battula
> >>>>
> >>>
> >>
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message