spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Pentreath" <nick.pentre...@gmail.com>
Subject Re: SparkException: Expect only DirectTaskResults when using localScheduler()
Date Sat, 18 Jan 2014 08:08:16 GMT
Ok that makes sense thanks. What is the default all a frame size?—
Sent from Mailbox for iPhone

On Sat, Jan 18, 2014 at 9:54 AM, Patrick Wendell <pwendell@gmail.com>
wrote:

> This is a bug that was fixed and will be part of 0.8.2:
> https://github.com/apache/incubator-spark/pull/281
> A workaround is setting the akka frame size to be larger using
> spark.akka.frameSize.
> The issue is that we added a mechanism to circumvent akka for large
> task results, but the implementation assumed this would never happen
> in local mode. That assumption turned out to be wrong, so it was
> patched in #281.
> - Patrick
> On Fri, Jan 17, 2014 at 9:53 PM, Nick Pentreath
> <nick.pentreath@gmail.com> wrote:
>> I'm also running into this issue. If it is related to a setting then the
>> error message should surely be cleaned up a bit to be more helpful in
>> suggesting a fix?
>> —
>> Sent from Mailbox for iPhone
>>
>>
>> On Sat, Jan 18, 2014 at 2:35 AM, Hussam_Jarada@dell.com
>> <Hussam_Jarada@dell.com> wrote:
>>>
>>> Hi,
>>>
>>>
>>>
>>> Can someone please explain what it means the following exception
>>> ‘SparkException: Expect only DirectTaskResults when using localScheduler()’?
>>>
>>> This is related to spark.akka.frameSize?
>>>
>>> I am running single node local spark 0.8.1
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Hussam
>>
>>
Mime
View raw message