spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 1.0.1 (RC2)
Date Fri, 11 Jul 2014 16:46:09 GMT
Okay just FYI - I'm closing this vote since many people are waiting on
the release and I was hoping to package it today. If we find a
reproducible Mesos issue here, we can definitely spin the fix into a
subsequent release.



On Fri, Jul 11, 2014 at 9:37 AM, Patrick Wendell <pwendell@gmail.com> wrote:
> Hey Gary,
>
> Why do you think the akka frame size changed? It didn't change - we
> added some fixes for cases where users were setting non-default
> values.
>
> On Fri, Jul 11, 2014 at 9:31 AM, Gary Malouf <malouf.gary@gmail.com> wrote:
>> Hi Matei,
>>
>> We have not had time to re-deploy the rc today, but one thing that jumps
>> out is the shrinking of the default akka frame size from 10MB to around
>> 128KB by default.  That is my first suspicion for our issue - could imagine
>> that biting others as well.
>>
>> I'll try to re-test that today - either way, understand moving forward at
>> this point.
>>
>> Gary
>>
>>
>> On Fri, Jul 11, 2014 at 12:08 PM, Matei Zaharia <matei.zaharia@gmail.com>
>> wrote:
>>
>>> Unless you can diagnose the problem quickly, Gary, I think we need to go
>>> ahead with this release as is. This release didn't touch the Mesos support
>>> as far as I know, so the problem might be a nondeterministic issue with
>>> your application. But on the other hand the release does fix some critical
>>> bugs that affect all users. We can always do 1.0.2 later if we discover a
>>> problem.
>>>
>>> Matei
>>>
>>> On Jul 10, 2014, at 9:40 PM, Patrick Wendell <pwendell@gmail.com> wrote:
>>>
>>> > Hey Gary,
>>> >
>>> > The vote technically doesn't close until I send the vote summary
>>> > e-mail, but I was planning to close and package this tonight. It's too
>>> > bad if there is a regression, it might be worth holding the release
>>> > but it really requires narrowing down the issue to get more
>>> > information about the scope and severity. Could you fork another
>>> > thread for this?
>>> >
>>> > - Patrick
>>> >
>>> > On Thu, Jul 10, 2014 at 6:28 PM, Gary Malouf <malouf.gary@gmail.com>
>>> wrote:
>>> >> -1 I honestly do not know the voting rules for the Spark community,
so
>>> >> please excuse me if I am out of line or if Mesos compatibility is not
a
>>> >> concern at this point.
>>> >>
>>> >> We just tried to run this version built against 2.3.0-cdh5.0.2 on mesos
>>> >> 0.18.2.  All of our jobs with data above a few gigabytes hung
>>> indefinitely.
>>> >> Downgrading back to the 1.0.0 stable release of Spark built the same
way
>>> >> worked for us.
>>> >>
>>> >>
>>> >> On Mon, Jul 7, 2014 at 5:17 PM, Tom Graves <tgraves_cs@yahoo.com.invalid
>>> >
>>> >> wrote:
>>> >>
>>> >>> +1. Ran some Spark on yarn jobs on a hadoop 2.4 cluster with
>>> >>> authentication on.
>>> >>>
>>> >>> Tom
>>> >>>
>>> >>>
>>> >>> On Friday, July 4, 2014 2:39 PM, Patrick Wendell <pwendell@gmail.com>
>>> >>> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> Please vote on releasing the following candidate as Apache Spark
>>> version
>>> >>> 1.0.1!
>>> >>>
>>> >>> The tag to be voted on is v1.0.1-rc1 (commit 7d1043c):
>>> >>>
>>> >>>
>>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7d1043c99303b87aef8ee19873629c2bfba4cc78
>>> >>>
>>> >>> The release files, including signatures, digests, etc. can be found
at:
>>> >>> http://people.apache.org/~pwendell/spark-1.0.1-rc2/
>>> >>>
>>> >>> Release artifacts are signed with the following key:
>>> >>> https://people.apache.org/keys/committer/pwendell.asc
>>> >>>
>>> >>> The staging repository for this release can be found at:
>>> >>>
>>> https://repository.apache.org/content/repositories/orgapachespark-1021/
>>> >>>
>>> >>> The documentation corresponding to this release can be found at:
>>> >>> http://people.apache.org/~pwendell/spark-1.0.1-rc2-docs/
>>> >>>
>>> >>> Please vote on releasing this package as Apache Spark 1.0.1!
>>> >>>
>>> >>> The vote is open until Monday, July 07, at 20:45 UTC and passes
if
>>> >>> a majority of at least 3 +1 PMC votes are cast.
>>> >>>
>>> >>> [ ] +1 Release this package as Apache Spark 1.0.1
>>> >>> [ ] -1 Do not release this package because ...
>>> >>>
>>> >>> To learn more about Apache Spark, please see
>>> >>> http://spark.apache.org/
>>> >>>
>>> >>> === Differences from RC1 ===
>>> >>> This release includes only one "blocking" patch from rc1:
>>> >>> https://github.com/apache/spark/pull/1255
>>> >>>
>>> >>> There are also smaller fixes which came in over the last week.
>>> >>>
>>> >>> === About this release ===
>>> >>> This release fixes a few high-priority bugs in 1.0 and has a variety
>>> >>> of smaller fixes. The full list is here: http://s.apache.org/b45.
Some
>>> >>> of the more visible patches are:
>>> >>>
>>> >>> SPARK-2043: ExternalAppendOnlyMap doesn't always find matching keys
>>> >>> SPARK-2156 and SPARK-1112: Issues with jobs hanging due to akka
frame
>>> size.
>>> >>> SPARK-1790: Support r3 instance types on EC2.
>>> >>>
>>> >>> This is the first maintenance release on the 1.0 line. We plan to
make
>>> >>> additional maintenance releases as new fixes come in.
>>> >>>
>>>
>>>

Mime
View raw message