spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Zhen <zhpeng...@gmail.com>
Subject Re: SparkSQL exception on spark.sql.codegen
Date Wed, 19 Nov 2014 01:27:29 GMT
Okay, thank you Micheal.

On Wed, Nov 19, 2014 at 3:45 AM, Michael Armbrust <michael@databricks.com>
wrote:

> Those are probably related.  It looks like we are somehow not being thread
> safe when initializing various parts of the scala compiler.
>
> Since code gen is pretty experimental we probably won't have the resources
> to investigate backporting a fix.  However, if you can reproduce the
> problem in Spark 1.2 then please file a JIRA.
>
> On Mon, Nov 17, 2014 at 9:37 PM, Eric Zhen <zhpeng.is@gmail.com> wrote:
>
>> Yes, it's always appears on a part of the whole tasks in a stage(i.e. 100/100
>> (65 failed)), and sometimes cause the stage to fail.
>>
>> And there is another error that I'm not sure if there is a correlation.
>>
>> java.lang.NoClassDefFoundError: Could not initialize class
>> org.apache.spark.sql.catalyst.expressions.codegen.GeneratePredicate$
>>         at
>> org.apache.spark.sql.execution.SparkPlan.newPredicate(SparkPlan.scala:114)
>>         at
>> org.apache.spark.sql.execution.Filter.conditionEvaluator$lzycompute(basicOperators.scala:55)
>>         at
>> org.apache.spark.sql.execution.Filter.conditionEvaluator(basicOperators.scala:55)
>>         at
>> org.apache.spark.sql.execution.Filter$$anonfun$2.apply(basicOperators.scala:58)
>>         at
>> org.apache.spark.sql.execution.Filter$$anonfun$2.apply(basicOperators.scala:57)
>>         at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596)
>>         at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596)
>>         at
>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>>         at
>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>>         at
>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>>         at
>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
>>         at
>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>>         at org.apache.spark.scheduler.Task.run(Task.scala:54)
>>         at
>> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178)
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>         at java.lang.Thread.run(Thread.java:744)
>>
>> On Tue, Nov 18, 2014 at 11:41 AM, Michael Armbrust <
>> michael@databricks.com> wrote:
>>
>>> Interesting, I believe we have run that query with version 1.1.0 with
>>> codegen turned on and not much has changed there.  Is the error
>>> deterministic?
>>>
>>> On Mon, Nov 17, 2014 at 7:04 PM, Eric Zhen <zhpeng.is@gmail.com> wrote:
>>>
>>>> Hi Michael,
>>>>
>>>> We use Spark v1.1.1-rc1 with jdk 1.7.0_51 and scala 2.10.4.
>>>>
>>>> On Tue, Nov 18, 2014 at 7:09 AM, Michael Armbrust <
>>>> michael@databricks.com> wrote:
>>>>
>>>>> What version of Spark SQL?
>>>>>
>>>>> On Sat, Nov 15, 2014 at 10:25 PM, Eric Zhen <zhpeng.is@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> We run SparkSQL on TPCDS benchmark Q19 with  spark.sql.codegen=true,
>>>>>> we got exceptions as below, has anyone else saw these before?
>>>>>>
>>>>>> java.lang.ExceptionInInitializerError
>>>>>>         at
>>>>>> org.apache.spark.sql.execution.SparkPlan.newProjection(SparkPlan.scala:92)
>>>>>>         at
>>>>>> org.apache.spark.sql.execution.Exchange$$anonfun$execute$1$$anonfun$1.apply(Exchange.scala:51)
>>>>>>         at
>>>>>> org.apache.spark.sql.execution.Exchange$$anonfun$execute$1$$anonfun$1.apply(Exchange.scala:48)
>>>>>>         at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596)
>>>>>>         at org.apache.spark.rdd.RDD$$anonfun$13.apply(RDD.scala:596)
>>>>>>         at
>>>>>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>>>>>         at
>>>>>> org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>>>>>>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>>>>>>         at
>>>>>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
>>>>>>         at
>>>>>> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>>>>>>         at org.apache.spark.scheduler.Task.run(Task.scala:54)
>>>>>>         at
>>>>>> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178)
>>>>>>         at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>>         at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>>         at java.lang.Thread.run(Thread.java:744)
>>>>>> Caused by: java.lang.NullPointerException
>>>>>>         at
>>>>>> scala.reflect.internal.Types$TypeRef.computeHashCode(Types.scala:2358)
>>>>>>         at
>>>>>> scala.reflect.internal.Types$UniqueType.<init>(Types.scala:1304)
>>>>>>         at
>>>>>> scala.reflect.internal.Types$TypeRef.<init>(Types.scala:2341)
>>>>>>         at
>>>>>> scala.reflect.internal.Types$NoArgsTypeRef.<init>(Types.scala:2137)
>>>>>>         at
>>>>>> scala.reflect.internal.Types$TypeRef$$anon$6.<init>(Types.scala:2544)
>>>>>>         at
>>>>>> scala.reflect.internal.Types$TypeRef$.apply(Types.scala:2544)
>>>>>>         at
>>>>>> scala.reflect.internal.Types$class.typeRef(Types.scala:3615)
>>>>>>         at
>>>>>> scala.reflect.internal.SymbolTable.typeRef(SymbolTable.scala:13)
>>>>>>         at
>>>>>> scala.reflect.internal.Symbols$TypeSymbol.newTypeRef(Symbols.scala:2752)
>>>>>>         at
>>>>>> scala.reflect.internal.Symbols$TypeSymbol.typeConstructor(Symbols.scala:2806)
>>>>>>         at
>>>>>> scala.reflect.internal.Symbols$SymbolContextApiImpl.toTypeConstructor(Symbols.scala:103)
>>>>>>         at
>>>>>> scala.reflect.internal.Symbols$TypeSymbol.toTypeConstructor(Symbols.scala:2698)
>>>>>>         at
>>>>>> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$typecreator1$1.apply(CodeGenerator.scala:46)
>>>>>>         at
>>>>>> scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:231)
>>>>>>         at
>>>>>> scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:231)
>>>>>>         at scala.reflect.api.TypeTags$class.typeOf(TypeTags.scala:335)
>>>>>>         at scala.reflect.api.Universe.typeOf(Universe.scala:59)
>>>>>>         at
>>>>>> org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator.<init>(CodeGenerator.scala:46)
>>>>>>         at
>>>>>> org.apache.spark.sql.catalyst.expressions.codegen.GenerateProjection$.<init>(GenerateProjection.scala:29)
>>>>>>         at
>>>>>> org.apache.spark.sql.catalyst.expressions.codegen.GenerateProjection$.<clinit>(GenerateProjection.scala)
>>>>>>         ... 15 more
>>>>>> --
>>>>>> Best Regards
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>
>>>
>>
>>
>> --
>> Best Regards
>>
>
>


-- 
Best Regards

Mime
View raw message