hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hung <jyhung2...@gmail.com>
Subject Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)
Date Sat, 26 Oct 2019 17:29:36 GMT
Hi Eric, I took a quick look, are you using
mapreduce.application.framework.path to run your MR jobs? If not, this
seems like expected behavior if AM and tasks get launched on different NMs
with different locally installed hadoop versions?

Jonathan Hung


On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <epayne@apache.org> wrote:

> I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)
>
> Unfortunately, I ran into the following problem:
>
> Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the
> following error:
>
> 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch):
> Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch.
> (client = 19, server = 21)
>
> The AM happened to launch on the 3.3.0 node.
>
> Is this a protobuf issue? I thought we addressed that?
>
> -Eric Payne
>
>
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <ebadger@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jyhung2357@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <ztang@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jyhung2357@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to
3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23
at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message