spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Chang <m...@tellapart.com>
Subject Re: Failed to remove RDD error
Date Mon, 09 Jun 2014 17:06:08 GMT
Unfortunately don't have any logs at the moment.  I will post them here if
they occur again!


On Tue, Jun 3, 2014 at 11:04 AM, Tathagata Das <tathagata.das1565@gmail.com>
wrote:

> It was not intended to be experimental as this improves general
> performance. We tested the feature since 0.9, and didnt see any problems.
> We need to investigate the cause of this. Can you give us the logs showing
> this error so that we can analyze this.
>
> TD
>
>
> On Tue, Jun 3, 2014 at 10:08 AM, Michael Chang <mike@tellapart.com> wrote:
>
>> Thanks Tathagata,
>>
>> Thanks for all your hard work!  In the future, is it possible to mark
>> "experimental" features as such on the online documentation?
>>
>> Thanks,
>> Michael
>>
>>
>> On Mon, Jun 2, 2014 at 6:12 PM, Tathagata Das <
>> tathagata.das1565@gmail.com> wrote:
>>
>>> Spark.streaming.unpersist was an experimental feature introduced with
>>> Spark 0.9 (but kept disabled), which actively clears off RDDs that are not
>>> useful any more. in Spark 1.0 that has been enabled by default. It is
>>> possible that this is an unintended side-effect of that. If
>>> spark.cleaner.ttl works then that should be used.
>>>
>>> TD
>>>
>>>
>>> On Mon, Jun 2, 2014 at 9:42 AM, Michael Chang <mike@tellapart.com>
>>> wrote:
>>>
>>>> Hey Mayur,
>>>>
>>>> Thanks for the suggestion, I didn't realize that was configurable.  I
>>>> don't think I'm running out of memory, though it does seem like these
>>>> errors go away when i turn off the spark.streaming.unpersist configuration
>>>> and use spark.cleaner.ttl instead.  Do you know if there are known issues
>>>> with the unpersist option?
>>>>
>>>>
>>>> On Sat, May 31, 2014 at 12:17 AM, Mayur Rustagi <
>>>> mayur.rustagi@gmail.com> wrote:
>>>>
>>>>> You can increase your akka timeout, should give you some more life..
>>>>> are you running out of memory by any chance?
>>>>>
>>>>>
>>>>> Mayur Rustagi
>>>>> Ph: +1 (760) 203 3257
>>>>> http://www.sigmoidanalytics.com
>>>>> @mayur_rustagi <https://twitter.com/mayur_rustagi>
>>>>>
>>>>>
>>>>>
>>>>> On Sat, May 31, 2014 at 6:52 AM, Michael Chang <mike@tellapart.com>
>>>>> wrote:
>>>>>
>>>>>> I'm running a some kafka streaming spark contexts (on 0.9.1), and
>>>>>> they seem to be dying after 10 or so minutes with a lot of these
errors.  I
>>>>>> can't really tell what's going on here, except that maybe the driver
is
>>>>>> unresponsive somehow?  Has anyone seen this before?
>>>>>>
>>>>>> 14/05/31 01:13:30 ERROR BlockManagerMaster: Failed to remove RDD
12635
>>>>>>
>>>>>> akka.pattern.AskTimeoutException: Timed out
>>>>>>
>>>>>>         at
>>>>>> akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334)
>>>>>>
>>>>>>         at akka.actor.Scheduler$$anon$11.run(Scheduler.scala:118)
>>>>>>
>>>>>>         at
>>>>>> scala.concurrent.Future$InternalCallbackExecutor$.scala$concurrent$Future$InternalCallbackExecutor$$unbatchedExecute(Future.scala:691)
>>>>>>
>>>>>>         at
>>>>>> scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:688)
>>>>>>
>>>>>>         at
>>>>>> akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:455)
>>>>>>
>>>>>>         at
>>>>>> akka.actor.LightArrayRevolverScheduler$$anon$12.executeBucket$1(Scheduler.scala:407)
>>>>>>
>>>>>>         at
>>>>>> akka.actor.LightArrayRevolverScheduler$$anon$12.nextTick(Scheduler.scala:411)
>>>>>>
>>>>>>         at
>>>>>> akka.actor.LightArrayRevolverScheduler$$anon$12.run(Scheduler.scala:363)
>>>>>>
>>>>>>         at java.lang.Thread.run(Thread.java:744)
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message