spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject [RESULT] [VOTE] Release Apache Spark 1.3.1
Date Wed, 08 Apr 2015 05:38:57 GMT
This vote is cancelled in favor of RC2.

On Tue, Apr 7, 2015 at 8:13 PM, Josh Rosen <rosenville@gmail.com> wrote:
> The leak will impact long running streaming jobs even if they don't write Hadoop files,
although the problem may take much longer to manifest itself for those jobs.
>
> I think we currently leak an empty HashMap per stage submitted in the common case, so
it could take a very long time for this to trigger an OOM.  On the other hand, the worst case
behavior is quite bad for streaming jobs, so we should probably fix this so that 1.2.x streaming
users can more safely upgrade to 1.3.x.
>
> - Josh
>
> Sent from my phone
>
>> On Apr 7, 2015, at 4:13 PM, Patrick Wendell <pwendell@gmail.com> wrote:
>>
>> Hey All,
>>
>> Today SPARK-6737 came to my attention. This is a bug that causes a
>> memory leak for any long running program that repeatedly saves data
>> out to a Hadoop FileSystem. For that reason, it is problematic for
>> Spark Streaming.
>>
>> My sense is that this is severe enough to cut another RC once the fix
>> is merged (which is imminent):
>>
>> https://issues.apache.org/jira/browse/SPARK-6737
>>
>> I'll leave a bit of time for others to comment, in particular if
>> people feel we should not wait for this fix.
>>
>> - Patrick
>>
>>> On Tue, Apr 7, 2015 at 2:34 PM, Marcelo Vanzin <vanzin@cloudera.com> wrote:
>>> +1 (non-binding)
>>>
>>> Ran standalone and yarn tests on the hadoop-2.6 tarball, with and
>>> without the external shuffle service in yarn mode.
>>>
>>>> On Sat, Apr 4, 2015 at 5:09 PM, Patrick Wendell <pwendell@gmail.com>
wrote:
>>>> Please vote on releasing the following candidate as Apache Spark version
1.3.1!
>>>>
>>>> The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f):
>>>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851
>>>>
>>>> The list of fixes present in this release can be found at:
>>>> http://bit.ly/1C2nVPY
>>>>
>>>> The release files, including signatures, digests, etc. can be found at:
>>>> http://people.apache.org/~pwendell/spark-1.3.1-rc1/
>>>>
>>>> Release artifacts are signed with the following key:
>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>
>>>> The staging repository for this release can be found at:
>>>> https://repository.apache.org/content/repositories/orgapachespark-1080
>>>>
>>>> The documentation corresponding to this release can be found at:
>>>> http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/
>>>>
>>>> Please vote on releasing this package as Apache Spark 1.3.1!
>>>>
>>>> The vote is open until Wednesday, April 08, at 01:10 UTC and passes
>>>> if a majority of at least 3 +1 PMC votes are cast.
>>>>
>>>> [ ] +1 Release this package as Apache Spark 1.3.1
>>>> [ ] -1 Do not release this package because ...
>>>>
>>>> To learn more about Apache Spark, please see
>>>> http://spark.apache.org/
>>>>
>>>> - Patrick
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>
>>>
>>>
>>> --
>>> Marcelo
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message