hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billie Rinaldi <bil...@apache.org>
Subject Re: [DISCUSS] Hadoop 3.3.1 release
Date Mon, 19 Apr 2021 18:44:05 GMT
I was thinking of backporting HADOOP-16948 for 3.3.1.

Billie

On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
<weichiu@cloudera.com.invalid> wrote:

> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vinayakumarb@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <dam6923@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable
and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is
a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming
3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based
on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <stbilwa@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We
have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and
tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the
same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula
<
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me
know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be
included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan
<
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for
3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack
<stack@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve
Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c:
how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means
a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people
who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out
(reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48,
Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would
need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using
the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at
6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > ´╗┐Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was
released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes
in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for
3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the
release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are
 currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >   1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > >   2.
> > > > > > >> >> > > > > >      1. HADOOP-17112
<
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >         2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who
would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease
wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important
for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands,
e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message