hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gera Shegalov <g...@apache.org>
Subject Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
Date Tue, 03 Apr 2018 21:07:31 GMT
+1 (non-binding)

- built from source
- tested SparkPi on minicluster (modulo YARN-7747
<https://issues.apache.org/jira/browse/YARN-7747>)
- tested SparkPi on pseudo-distributed cluster
- browsed HDFS doc in the site tarball


On Tue, Apr 3, 2018 at 11:55 AM Hanisha Koneru <hkoneru@hortonworks.com>
wrote:

> Correction: My vote is NON-BINDING. Sorry for the confusion.
>
>
> Thanks,
> Hanisha
>
>
>
>
>
>
>
>
> On 4/3/18, 11:40 AM, "Hanisha Koneru" <hkoneru@hortonworks.com> wrote:
>
> >Thanks Wangda for putting up the RC for 3.1.0.
> >
> >+1 (binding).
> >
> >Verified the following:
> >- Built from source
> >- Deployed binary to a 3-node docker cluster
> >- Sanity checks
> >       - Basic dfs operations
> >       - MapReduce Wordcount & Grep
> >
> >
> >Thanks,
> >Hanisha
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >On 4/3/18, 9:33 AM, "Arpit Agarwal" <aagarwal@hortonworks.com> wrote:
> >
> >>Thanks Wangda, I see the shaded jars now.
> >>
> >>Are the repo jars required to be the same as the binary release? They
> don’t match right now, probably they got rebuilt.
> >>
> >>+1 (binding), modulo that remaining question.
> >>
> >>* Verified signatures
> >>* Verified checksums for source and binary artefacts
> >>* Sanity checked jars on r.a.o.
> >>* Built from source
> >>* Deployed to 3 node secure cluster with NameNode HA
> >>* Verified HDFS web UIs
> >>* Tried out HDFS shell commands
> >>* Ran sample MapReduce jobs
> >>
> >>Thanks!
> >>
> >>
> >>----------------------------------------------------------------------
> >>From: Wangda Tan <wheeleast@gmail.com>
> >>Date: Monday, April 2, 2018 at 9:25 PM
> >>To: Arpit Agarwal <aagarwal@hortonworks.com>
> >>Cc: Gera Shegalov <gera05@gmail.com>, Sunil G <sunilg@apache.org>,
"
> yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>, Hdfs-dev <
> hdfs-dev@hadoop.apache.org>, Hadoop Common <common-dev@hadoop.apache.org>,
> "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
> Vinod Kumar Vavilapalli <vinodkv@apache.org>
> >>Subject: Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
> >>
> >>As pointed by Arpit, the previously deployed shared jars are incorrect.
> Just redeployed jars and staged. @Arpit, could you please check the updated
> Maven repo?
> https://repository.apache.org/content/repositories/orgapachehadoop-1092
> >>
> >>Since the jars inside binary tarballs are correct (
> http://people.apache.org/~wangda/hadoop-3.1.0-RC1/). I think we don't
> need roll another RC, just update Maven repo should be sufficient.
> >>
> >>Best,
> >>Wangda
> >>
> >>
> >>On Mon, Apr 2, 2018 at 2:39 PM, Wangda Tan <mailto:wheeleast@gmail.com>
> wrote:
> >>Hi Arpit,
> >>
> >>Thanks for pointing out this.
> >>
> >>I just removed all .md5 files from artifacts. I found md5 checksums
> still exist in .mds files and I didn't remove them from .mds file because
> it is generated by create-release script and Apache guidance is "should
> not" instead of "must not". Please let me know if you think they need to be
> removed as well.
> >>
> >>- Wangda
> >>
> >>
> >>
> >>On Mon, Apr 2, 2018 at 1:37 PM, Arpit Agarwal <mailto:
> aagarwal@hortonworks.com> wrote:
> >>Thanks for putting together this RC, Wangda.
> >>
> >>The guidance from Apache is to omit MD5s, specifically:
> >>  > SHOULD NOT supply a MD5 checksum file (because MD5 is too broken).
> >>
> >>https://www.apache.org/dev/release-distribution#sigs-and-sums
> >>
> >>
> >>
> >>
> >>On Apr 2, 2018, at 7:03 AM, Wangda Tan <mailto:wheeleast@gmail.com>
> wrote:
> >>
> >>Hi Gera,
> >>
> >>It's my bad, I thought only src/bin tarball is enough.
> >>
> >>I just uploaded all other things under artifact/ to
> >>http://people.apache.org/~wangda/hadoop-3.1.0-RC1/
> >>
> >>Please let me know if you have any other comments.
> >>
> >>Thanks,
> >>Wangda
> >>
> >>
> >>On Mon, Apr 2, 2018 at 12:50 AM, Gera Shegalov <mailto:gera05@gmail.com>
> wrote:
> >>
> >>
> >>Thanks, Wangda!
> >>
> >>There are many more artifacts in previous votes, e.g., see
> >>http://home.apache.org/~junping_du/hadoop-2.8.3-RC0/ .  Among others the
> >>site tarball is missing.
> >>
> >>On Sun, Apr 1, 2018 at 11:54 PM Sunil G <mailto:sunilg@apache.org>
> wrote:
> >>
> >>
> >>Thanks Wangda for initiating the release.
> >>
> >>I tested this RC built from source file.
> >>
> >>
> >>  - Tested MR apps (sleep, wc) and verified both new YARN UI and old RM
> >>UI.
> >>  - Below feature sanity is done
> >>     - Application priority
> >>     - Application timeout
> >>     - Intra Queue preemption with priority based
> >>     - DS based affinity tests to verify placement constraints.
> >>  - Tested basic NodeLabel scenarios.
> >>     - Added couple of labels to few of nodes and behavior is coming
> >>     correct.
> >>     - Verified old UI  and new YARN UI for labels.
> >>     - Submitted apps to labelled cluster and it works fine.
> >>     - Also performed few cli commands related to nodelabel.
> >>  - Test basic HA cases and seems correct.
> >>  - Tested new YARN UI . All pages are getting loaded correctly.
> >>
> >>
> >>- Sunil
> >>
> >>On Fri, Mar 30, 2018 at 9:45 AM Wangda Tan <mailto:wheeleast@gmail.com>
> wrote:
> >>
> >>
> >>Hi folks,
> >>
> >>Thanks to the many who helped with this release since Dec 2017 [1].
> >>We've
> >>
> >>created RC1 for Apache Hadoop 3.1.0. The artifacts are available here:
> >>
> >>http://people.apache.org/~wangda/hadoop-3.1.0-RC1
> >>
> >>The RC tag in git is release-3.1.0-RC1. Last git commit SHA is
> >>16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> >>
> >>The maven artifacts are available via http://repository.apache.org at
> >>https://repository.apache.org/content/repositories/
> >>orgapachehadoop-1090/
> >>
> >>This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific.
> >>
> >>3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions
> >>include the first class GPU/FPGA support on YARN, Native services,
> >>Support
> >>
> >>rich placement constraints in YARN, S3-related enhancements, allow HDFS
> >>block replicas to be provided by an external storage system, etc.
> >>
> >>For 3.1.0 RC0 vote discussion, please see [3].
> >>
> >>We’d like to use this as a starting release for 3.1.x [1], depending on
> >>how
> >>
> >>it goes, get it stabilized and potentially use a 3.1.1 in several weeks
> >>as
> >>
> >>the stable release.
> >>
> >>We have done testing with a pseudo cluster:
> >>- Ran distributed job.
> >>- GPU scheduling/isolation.
> >>- Placement constraints (intra-application anti-affinity) by using
> >>distributed shell.
> >>
> >>My +1 to start.
> >>
> >>Best,
> >>Wangda/Vinod
> >>
> >>[1]
> >>
> >>https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b
> >>c9e19e559aedc5ebedb4ca08c8@%http://3Cyarn-dev.hadoop.apache.org%3E
> >>
> >>[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0)
> >>AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER
> >>BY
> >>
> >>fixVersion ASC
> >>[3]
> >>
> >>https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7
> >>2d4061f26f83547e41d0983ea6@%http://3Cyarn-dev.hadoop.apache.org%3E
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message