hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <bra...@apache.org>
Subject Re: [DISCUSS] Hadoop 3.3.0 Release include ARM binary
Date Thu, 04 Jun 2020 14:43:02 GMT
Following blocker is pending for 3.3.0 release which is ready for review.
Mostly we'll have RC soon.
https://issues.apache.org/jira/browse/HADOOP-17046

Protobuf dependency was unexpected .

On Mon, Jun 1, 2020 at 7:11 AM Sheng Liu <liusheng2048@gmail.com> wrote:

> Hi folks,
>
> It looks like the 3.3.0 branch has been created for quite a while. Not sure
> if there is remain block issue that need to be addressed before Hadoop
> 3.3.0 release publishing, maybe we can bring up to here and move the
> release forward ?
>
> Thank.
>
> Brahma Reddy Battula <brahma@apache.org> 于2020年3月25日周三 上午1:55写道:
>
> > thanks to all.
> >
> > will make this as optional..will update the wiki accordingly.
> >
> > On Wed, Mar 18, 2020 at 12:05 AM Vinayakumar B <vinayakumarb@apache.org>
> > wrote:
> >
> > > Making ARM artifact optional, makes the release process simpler for RM
> > and
> > > unblocks release process (if there is unavailability of ARM resources).
> > >
> > > Still there are possible options to collaborate with RM ( as brahma
> > > mentioned earlier) and provide ARM artifact may be before or after
> vote.
> > > If feasible RM can decide to add ARM artifact by collaborating with
> > @Brahma
> > > Reddy Battula <brahma@apache.org> or me to get the ARM artifact.
> > >
> > > -Vinay
> > >
> > > On Tue, Mar 17, 2020 at 11:39 PM Arpit Agarwal
> > > <aagarwal@cloudera.com.invalid> wrote:
> > >
> > > > Thanks for the clarification Brahma. Can you update the proposal to
> > state
> > > > that it is optional (it may help to put the proposal on cwiki)?
> > > >
> > > > Also if we go ahead then the RM documentation should be clear this is
> > an
> > > > optional step.
> > > >
> > > >
> > > > > On Mar 17, 2020, at 11:06 AM, Brahma Reddy Battula <
> > brahma@apache.org>
> > > > wrote:
> > > > >
> > > > > Sure, we can't make mandatory while voting and we can upload to
> > > downloads
> > > > > once release vote is passed.
> > > > >
> > > > > On Tue, 17 Mar 2020 at 11:24 PM, Arpit Agarwal
> > > > > <aagarwal@cloudera.com.invalid> wrote:
> > > > >
> > > > >>> Sorry,didn't get you...do you mean, once release voting is
> > > > >>> processed and upload by RM..?
> > > > >>
> > > > >> Yes, that is what I meant. I don’t want us to make more mandatory
> > work
> > > > for
> > > > >> the release manager because the job is hard enough already.
> > > > >>
> > > > >>
> > > > >>> On Mar 17, 2020, at 10:46 AM, Brahma Reddy Battula <
> > > brahma@apache.org>
> > > > >> wrote:
> > > > >>>
> > > > >>> Sorry,didn't get you...do you mean, once release voting is
> > processed
> > > > and
> > > > >>> upload by RM..?
> > > > >>>
> > > > >>> FYI. There is docker image for ARM also which support all
scripts
> > > > >>> (createrelease, start-build-env.sh, etc ).
> > > > >>>
> > > > >>> https://issues.apache.org/jira/browse/HADOOP-16797
> > > > >>>
> > > > >>> On Tue, Mar 17, 2020 at 10:59 PM Arpit Agarwal
> > > > >>> <aagarwal@cloudera.com.invalid> wrote:
> > > > >>>
> > > > >>>> Can ARM binaries be provided after the fact? We cannot
increase
> > the
> > > > RM’s
> > > > >>>> burden by asking them to generate an extra set of binaries.
> > > > >>>>
> > > > >>>>
> > > > >>>>> On Mar 17, 2020, at 10:23 AM, Brahma Reddy Battula
<
> > > > brahma@apache.org>
> > > > >>>> wrote:
> > > > >>>>>
> > > > >>>>> + Dev mailing list.
> > > > >>>>>
> > > > >>>>> ---------- Forwarded message ---------
> > > > >>>>> From: Brahma Reddy Battula <brahma@apache.org>
> > > > >>>>> Date: Tue, Mar 17, 2020 at 10:31 PM
> > > > >>>>> Subject: Re: [DISCUSS] Hadoop 3.3.0 Release include
ARM binary
> > > > >>>>> To: junping_du <junping_du@apache.org>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> thanks junping for your reply.
> > > > >>>>>
> > > > >>>>> bq.      I think most of us in Hadoop community doesn't
want to
> > > have
> > > > >>>> biased
> > > > >>>>> on ARM or any other platforms.
> > > > >>>>>
> > > > >>>>> Yes, release voting will be based on the source
> code.AFAIK,Binary
> > > we
> > > > >> are
> > > > >>>>> providing for user to easy to download and verify.
> > > > >>>>>
> > > > >>>>> bq.     The only thing I try to understand is how
much
> complexity
> > > get
> > > > >>>>> involved for our RM work. Does that potentially become
a
> blocker
> > > for
> > > > >>>> future
> > > > >>>>> releases? And how we can get rid of this risk.
> > > > >>>>>
> > > > >>>>> As I mentioned earlier, RM need to access the ARM
machine(it
> will
> > > be
> > > > >>>>> donated and current qbt also using one ARM machine)
and build
> tar
> > > > using
> > > > >>>> the
> > > > >>>>> keys. As it can be common machine, RM can delete
his keys once
> > > > release
> > > > >>>>> approved.
> > > > >>>>> Can be sorted out as I mentioned earlier.(For accessing
the ARM
> > > > >> machine)
> > > > >>>>>
> > > > >>>>> bq.       If you can list the concrete work that
RM need to do
> > > extra
> > > > >> for
> > > > >>>>> ARM release, that would help us to better understand.
> > > > >>>>>
> > > > >>>>> I can write and update for future reference.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Tue, Mar 17, 2020 at 10:41 AM 俊平堵 <junping_du@apache.org>
> > > wrote:
> > > > >>>>>
> > > > >>>>>> Hi Brahma,
> > > > >>>>>>   I think most of us in Hadoop community doesn't
want to have
> > > biased
> > > > >>>> on
> > > > >>>>>> ARM or any other platforms.
> > > > >>>>>>   The only thing I try to understand is how much
complexity
> get
> > > > >>>>>> involved for our RM work. Does that potentially
become a
> blocker
> > > for
> > > > >>>> future
> > > > >>>>>> releases? And how we can get rid of this risk.
> > > > >>>>>>    If you can list the concrete work that RM
need to do extra
> > for
> > > > ARM
> > > > >>>>>> release, that would help us to better understand.
> > > > >>>>>>
> > > > >>>>>> Thanks,
> > > > >>>>>>
> > > > >>>>>> Junping
> > > > >>>>>>
> > > > >>>>>> Akira Ajisaka <aajisaka@apache.org> 于2020年3月13日周五
上午12:34写道:
> > > > >>>>>>
> > > > >>>>>>> If you can provide ARM release for future
releases, I'm fine
> > with
> > > > >> that.
> > > > >>>>>>>
> > > > >>>>>>> Thanks,
> > > > >>>>>>> Akira
> > > > >>>>>>>
> > > > >>>>>>> On Thu, Mar 12, 2020 at 9:41 PM Brahma Reddy
Battula <
> > > > >>>> brahma@apache.org>
> > > > >>>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> thanks Akira.
> > > > >>>>>>>>
> > > > >>>>>>>> Currently only problem is dedicated ARM
for future RM.This i
> > > want
> > > > to
> > > > >>>>>>> sort
> > > > >>>>>>>> out like below,if you've some other,please
let me know.
> > > > >>>>>>>>
> > > > >>>>>>>> i) Single machine and share cred to future
RM ( as we can
> > delete
> > > > >> keys
> > > > >>>>>>> once
> > > > >>>>>>>> release is over).
> > > > >>>>>>>> ii) Creating the jenkins project ( may
be we need to discuss
> > in
> > > > the
> > > > >>>>>>>> board..)
> > > > >>>>>>>> iii) I can provide ARM release for future
releases.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Mar 12, 2020 at 5:14 PM Akira
Ajisaka <
> > > > aajisaka@apache.org>
> > > > >>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> Hi Brahma,
> > > > >>>>>>>>>
> > > > >>>>>>>>> I think we cannot do any of your
proposed actions.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> http://www.apache.org/legal/release-policy.html#owned-controlled-hardware
> > > > >>>>>>>>>> Strictly speaking, releases must
be verified on hardware
> > owned
> > > > and
> > > > >>>>>>>>> controlled by the committer. That
means hardware the
> > committer
> > > > has
> > > > >>>>>>>> physical
> > > > >>>>>>>>> possession and control of and exclusively
full
> > > > >>>>>>> administrative/superuser
> > > > >>>>>>>>> access to. That's because only such
hardware is qualified
> to
> > > > hold a
> > > > >>>>>>> PGP
> > > > >>>>>>>>> private key, and the release should
be verified on the
> > machine
> > > > the
> > > > >>>>>>>> private
> > > > >>>>>>>>> key lives on or on a machine as trusted
as that.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > https://www.apache.org/dev/release-distribution.html#sigs-and-sums
> > > > >>>>>>>>>> Private keys MUST NOT be stored
on any ASF machine.
> > Likewise,
> > > > >>>>>>>> signatures
> > > > >>>>>>>>> for releases MUST NOT be created
on ASF machines.
> > > > >>>>>>>>>
> > > > >>>>>>>>> We need to have dedicated physical
ARM machines for each
> > > release
> > > > >>>>>>> manager,
> > > > >>>>>>>>> and now it is not feasible.
> > > > >>>>>>>>> If you provide an unofficial ARM
binary release in some
> > > > repository,
> > > > >>>>>>>> that's
> > > > >>>>>>>>> okay.
> > > > >>>>>>>>>
> > > > >>>>>>>>> -Akira
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Thu, Mar 12, 2020 at 7:57 PM Brahma
Reddy Battula <
> > > > >>>>>>> brahma@apache.org>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Hello folks,
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> As currently trunk will support
ARM based compilation and
> > > qbt(1)
> > > > >> is
> > > > >>>>>>>>>> running
> > > > >>>>>>>>>> from several months with quite
stable, hence planning to
> > > propose
> > > > >> ARM
> > > > >>>>>>>>>> binary
> > > > >>>>>>>>>> this time.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> ( Note : As we'll know voting
will be based on the
> source,so
> > > > this
> > > > >>>>>>> will
> > > > >>>>>>>> not
> > > > >>>>>>>>>> issue.)
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> *Proposed Change:*
> > > > >>>>>>>>>> Currently in downloads we are
keeping only x86
> binary(2),Can
> > > we
> > > > >> keep
> > > > >>>>>>> ARM
> > > > >>>>>>>>>> binary also.?
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> *Actions:*
> > > > >>>>>>>>>> a) *Dedicated* *Machine*:
> > > > >>>>>>>>>>     i) Dedicated ARM machine
will be donated which I
> > confirmed
> > > > >>>>>>>>>>     ii) Or can use jenkins ARM
machine itself which is
> > > currently
> > > > >>>>>>> used
> > > > >>>>>>>>>> for ARM
> > > > >>>>>>>>>> b) *Automate Release:* How about
having one release
> project
> > in
> > > > >>>>>>>> jenkins..?
> > > > >>>>>>>>>> So that future RM's just trigger
the jenkin project.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Please let me know your thoughts
on this.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> 1.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-qbt-linux-ARM-trunk/
> > > > >>>>>>>>>> 2.https://hadoop.apache.org/releases.html
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> --Brahma Reddy Battula
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --Brahma Reddy Battula
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --Brahma Reddy Battula
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> --Brahma Reddy Battula
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > ---------------------------------------------------------------------
> > > > >>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > > >>>> For additional commands, e-mail:
> yarn-dev-help@hadoop.apache.org
> > > > >>>>
> > > > >>>>
> > > > >>>
> > > > >>> --
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> --Brahma Reddy Battula
> > > > >>
> > > > >>
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > >
> > > >
> > >
> >
> >
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> >
>


-- 



--Brahma Reddy Battula

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message