spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Date Mon, 20 Jan 2014 18:23:41 GMT
running the test cases, always block after passing “Interact with Files” in ReplSuite.scala

I checked the code, it is test("local-cluster mode") {

What happened here?  

--  
Nan Zhu


On Monday, January 20, 2014 at 1:15 PM, Reynold Xin wrote:

> That's a perm gen issue - you need to adjust the perm gem size. In sbt it should've been
set automatically, but I think for Maven, you need to set the maven opts, which is documented
in the build instructions.  
>  
>  
> On Sun, Jan 19, 2014 at 11:35 PM, Ewen Cheslack-Postava <ewencp@fastmail.fm (mailto:ewencp@fastmail.fm)>
wrote:
> > I can't get the tests to run on a Mac, 10.7.5, java -version output:
> >  
> > java version "1.6.0_65"
> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
> >  
> > For reference, Spark 0.8.* build and test find on the same configuration. 0.9.0-rc3
fails *after* PrimitiveVectorSuite, I'm not sure what it's running at that time since all
the tests in PrimitiveVectorSuite seem to have finished:
> >  
> > [info] PrimitiveVectorSuite:
> > [info] - primitive value (4 milliseconds)
> > [info] - non-primitive value (5 milliseconds)
> > [info] - ideal growth (4 milliseconds)
> > [info] - ideal size (5 milliseconds)
> > [info] - resizing (6 milliseconds)
> > [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4] [ActorSystem(spark)]
exception while executing timer task
> > org.apache.spark.SparkException: Error sending message to BlockManagerMaster [message
= HeartBeat(BlockManagerId(<driver>, localhost, 51634, 0))]
> >     at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176)
> >     at org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52)
> >     at org.apache.spark.storage.BlockManager.org (http://org.apache.spark.storage.BlockManager.org)$apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97)
> >     at org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135)
> >     at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80)
> >     at akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241)
> >     at akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464)
> >     at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281)
> >     at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280)
> >     at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> >     at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> >     at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> >     at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> >     at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279)
> >     at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630)
> >     at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582)
> >     at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
> >     at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
> >     at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596)
> >     at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750)
> >     at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753)
> >     at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
> >     at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
> >     at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15)
> >     at akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746)
> >     at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
> >     at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
> >     at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
> >     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
> >     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
> >     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
> >     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
> >     at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
> >     at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
> >     at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
> >     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
> >     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> >     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> >     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> >     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> > Caused by: akka.pattern.AskTimeoutException: Recipient[Actor[akka://spark/user/BlockManagerMaster#927284646]]
had already been terminated.
> >     at akka.pattern.AskableActorRef$.ask$extension(AskSupport.scala:134)
> >     at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:161)
> >     ... 39 more
> >  
> > [ERROR] [01/19/2014 23:20:03.131] [sparkWorker2-akka.actor.default-dispatcher-19]
[ActorSystem(sparkWorker2)] Uncaught fatal error from thread [sparkWorker2-akka.actor.default-dispatcher-19]
shutting down ActorSystem [sparkWorker2]
> > java.lang.OutOfMemoryError: PermGen space
> >  
> > [ERROR] [01/19/2014 23:20:05.226] [sparkWorker2-akka.actor.default-dispatcher-2]
[ActorSystem(sparkWorker2)] Uncaught fatal error from thread [sparkWorker2-akka.actor.default-dispatcher-2]
shutting down ActorSystem [sparkWorker2]
> > java.lang.OutOfMemoryError: PermGen space
> >  
> > [... snip ... followed by a bunch more like these ]
> >  
> > It looks like it's just a heap size issue. But since it's leaving a process running
that has -Xmx3g explicitly specified on the command line I'm surprised it works for everyone
else but consistently fails for me. Almost certainly not a blocker, but some memory setting
in the tests might need adjustment.
> >  
> > -Ewen
> > > Henry Saputra (mailto:henry.saputra@gmail.com)  
> > > January 19, 2014 10:07 PM
> > >  
> > >  
> > > Ah yes, makes sense, thanks!
> > >  
> > > - Henry
> > > Patrick Wendell (mailto:pwendell@gmail.com)  
> > > January 19, 2014 10:01 PM
> > >  
> > >  
> > > Eventually the notes get posted on the apache website. I attached them
> > > to this e-mail so that people can get a sense of what is in the
> > > release before they vote on it.
> > > Henry Saputra (mailto:henry.saputra@gmail.com)  
> > > January 19, 2014 9:57 PM
> > >  
> > >  
> > > Hi Patrick, quick question, where are you planning to add the release notes?
> > > I dont think it is part of the source, is it?
> > >  
> > > - Henry
> > > Patrick Wendell (mailto:pwendell@gmail.com)  
> > > January 19, 2014 8:41 PM
> > >  
> > >  
> > > Attempting to attach the release notes again (I think it may have been
> > > blocked previously due to not having an extension).
> > > Patrick Wendell (mailto:pwendell@gmail.com)  
> > > January 19, 2014 8:05 PM
> > >  
> > >  
> > > I'll add my +1 as well
>  


Mime
View raw message