spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc5)
Date Mon, 27 Jan 2014 06:58:40 GMT
Hey Taka,

If you build a second version you need to clean the existing assembly jar.

The reference implementation of the tests are the ones on the U.C.
Berkeley Jenkins. These are passing for Branch 0.9 for both Hadoop 1
and Hadoop 2 versions, so I'm inclined to think it's an issue with
your test env or setup.

https://amplab.cs.berkeley.edu/jenkins/view/Spark/

- Patrick

On Sun, Jan 26, 2014 at 10:52 PM, Reynold Xin <rxin@databricks.com> wrote:
> It is possible that you have generated the assembly jar using one version
> of Hadoop, and then another assembly jar with another version. Those tests
> that failed are all using a local cluster that sets up multiple processes,
> which would require launching Spark worker processes using the assembly
> jar. If that's indeed the problem, removing the extra assembly jars should
> fix them.
>
>
> On Sun, Jan 26, 2014 at 10:49 PM, Taka Shinagawa <taka.epsilon@gmail.com>wrote:
>
>> If I build Spark for Hadoop 1.0.4 (either "SPARK_HADOOP_VERSION=1.0.4
>> sbt/sbt assembly"  or "sbt/sbt assembly") or use the binary distribution,
>> 'sbt/sbt test' runs successfully.
>>
>> However, if I build Spark targeting any other Hadoop versions (e.g.
>> "SPARK_HADOOP_VERSION=1.2.1 sbt/sbt assembly", "SPARK_HADOOP_VERSION=2.2.0
>> sbt/sbt assembly"), I'm getting the following errors with 'sbt/sbt test':
>>
>> 1) type mismatch errors with JavaPairDStream.scala
>> 2) following test failures
>> [error] Failed tests:
>> [error] org.apache.spark.ShuffleNettySuite
>> [error] org.apache.spark.ShuffleSuite
>> [error] org.apache.spark.FileServerSuite
>> [error] org.apache.spark.DistributedSuite
>>
>> I don't have Hadoop 1.0.4 installed on my test systems (but the test
>> succeeds, and failed with the installed Hadoop versions). I'm seeing these
>> sbt test errors with the previous 0.9.0 RCs and 0.8.1, too.
>>
>> I'm wondering if anyone else has seen this problem or I'm missing something
>> to run the test correctly.
>>
>> Thanks,
>> Taka
>>
>>
>>
>>
>> On Sat, Jan 25, 2014 at 5:00 PM, Sean McNamara
>> <Sean.McNamara@webtrends.com>wrote:
>>
>> > +1
>> >
>> > On 1/25/14, 4:04 PM, "Mark Hamstra" <mark@clearstorydata.com> wrote:
>> >
>> > >+1
>> > >
>> > >
>> > >On Sat, Jan 25, 2014 at 2:37 PM, Andy Konwinski
>> > ><andykonwinski@gmail.com>wrote:
>> > >
>> > >> +1
>> > >>
>> > >>
>> > >> On Sat, Jan 25, 2014 at 2:27 PM, Reynold Xin <rxin@databricks.com>
>> > >>wrote:
>> > >>
>> > >> > +1
>> > >> >
>> > >> > > On Jan 25, 2014, at 12:07 PM, Hossein <falaki@gmail.com>
wrote:
>> > >> > >
>> > >> > > +1
>> > >> > >
>> > >> > > Compiled and tested on Mavericks.
>> > >> > >
>> > >> > > --Hossein
>> > >> > >
>> > >> > >
>> > >> > > On Sat, Jan 25, 2014 at 11:38 AM, Patrick Wendell
>> > >><pwendell@gmail.com
>> > >> > >wrote:
>> > >> > >
>> > >> > >> I'll kick of the voting with a +1.
>> > >> > >>
>> > >> > >> On Thu, Jan 23, 2014 at 11:33 PM, Patrick Wendell
>> > >><pwendell@gmail.com
>> > >> >
>> > >> > >> wrote:
>> > >> > >>> Please vote on releasing the following candidate
as Apache Spark
>> > >> > >>> (incubating) version 0.9.0.
>> > >> > >>>
>> > >> > >>> A draft of the release notes along with the changes
file is
>> > >>attached
>> > >> > >>> to this e-mail.
>> > >> > >>>
>> > >> > >>> The tag to be voted on is v0.9.0-incubating (commit
95d28ff3):
>> > >> > >>
>> > >> >
>> > >>
>> > >>
>> >
>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=
>> > >>95d28ff3d0d20d9c583e184f9e2c5ae842d8a4d9
>> > >> > >>>
>> > >> > >>> The release files, including signatures, digests,
etc can be
>> found
>> > >> at:
>> > >> > >>> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc5
>> > >> > >>>
>> > >> > >>> Release artifacts are signed with the following key:
>> > >> > >>> https://people.apache.org/keys/committer/pwendell.asc
>> > >> > >>>
>> > >> > >>> The staging repository for this release can be found
at:
>> > >> > >>>
>> > >> >
>> > >>
>> https://repository.apache.org/content/repositories/orgapachespark-1006/
>> > >> > >>>
>> > >> > >>> The documentation corresponding to this release can
be found at:
>> > >> > >>>
>> > >>http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc5-docs/
>> > >> > >>>
>> > >> > >>> Please vote on releasing this package as Apache Spark
>> > >> 0.9.0-incubating!
>> > >> > >>>
>> > >> > >>> The vote is open until Monday, January 27, at 07:30
UTC and
>> passes
>> > >> ifa
>> > >> > >>> majority of at least 3 +1 PPMC votes are cast.
>> > >> > >>>
>> > >> > >>> [ ] +1 Release this package as Apache Spark 0.9.0-incubating
>> > >> > >>> [ ] -1 Do not release this package because ...
>> > >> > >>>
>> > >> > >>> To learn more about Apache Spark, please see
>> > >> > >>> http://spark.incubator.apache.org/
>> > >> > >>
>> > >> >
>> > >> > --
>> > >> > You received this message because you are subscribed to the Google
>> > >>Groups
>> > >> > "Unofficial Apache Spark Dev Mailing List Mirror" group.
>> > >> > To unsubscribe from this group and stop receiving emails from
it,
>> > >>send an
>> > >> > email to apache-spark-dev-mirror+unsubscribe@googlegroups.com.
>> > >> > For more options, visit https://groups.google.com/groups/opt_out.
>> > >> >
>> > >>
>> >
>> >
>>

Mime
View raw message