spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: spark 2 segfault
Date Thu, 28 Apr 2016 18:41:29 GMT
Are you able to pastebin a unit test which can reproduce the following ?

Thanks

> On Apr 28, 2016, at 11:35 AM, Koert Kuipers <koert@tresata.com> wrote:
> 
> i tried for the first time to run our own in-house unit tests on spark 2, and i get the
error below.
> has anyone seen this?
> 
> it is reproducible. i tried latest java 7 and it is still there.
> 
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007f7c3a4b1f54, pid=21939, tid=140171011417856
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_75-b13) (build 1.7.0_75-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.75-b04 mixed mode linux-amd64 compressed
oops)
> # Problematic frame:
> # V  [libjvm.so+0x747f54]  _Copy_arrayof_conjoint_jlongs+0x44
> 
> more info:
> 
> Stack: [0x00007f7c1b47e000,0x00007f7c1b57f000],  sp=0x00007f7c1b57a9a8,  free space=1010k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x747f54]  _Copy_arrayof_conjoint_jlongs+0x44
> j  sun.misc.Unsafe.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V+0
> j  org.apache.spark.unsafe.Platform.copyMemory(Ljava/lang/Object;JLjava/lang/Object;JJ)V+34
> j  org.apache.spark.unsafe.types.UTF8String.getBytes()[B+76
> j  org.apache.spark.unsafe.types.UTF8String.toString()Ljava/lang/String;+5
> j  org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificSafeProjection.apply(Ljava/lang/Object;)Ljava/lang/Object;+876
> j  org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.fromRow(Lorg/apache/spark/sql/catalyst/InternalRow;)Ljava/lang/Object;+5
> j  org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$execute$1$1$$anonfun$apply$13.apply(Lorg/apache/spark/sql/catalyst/InternalRow;)Ljava/lang/Object;+11
> j  org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$execute$1$1$$anonfun$apply$13.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
> J 13277 C2 scala.collection.mutable.ArrayOps$ofRef.map(Lscala/Function1;Lscala/collection/generic/CanBuildFrom;)Ljava/lang/Object;
(7 bytes) @ 0x00007f7c25eeae08 [0x00007f7c25eead40+0xc8]
> j  org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$execute$1$1.apply()Ljava/lang/Object;+43
> j  org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(Lorg/apache/spark/sql/SparkSession;Lorg/apache/spark/sql/execution/QueryExecution;Lscala/Function0;)Ljava/lang/Object;+106
> j  org.apache.spark.sql.Dataset.withNewExecutionId(Lscala/Function0;)Ljava/lang/Object;+12
> j  org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$execute$1()Ljava/lang/Object;+9
> j  org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$collect$1.apply(Lorg/apache/spark/sql/Dataset;)Ljava/lang/Object;+4
> j  org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$collect$1.apply(Ljava/lang/Object;)Ljava/lang/Object;+5
> j  org.apache.spark.sql.Dataset.withCallback(Ljava/lang/String;Lorg/apache/spark/sql/Dataset;Lscala/Function1;)Ljava/lang/Object;+25
> j  org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$collect(Z)Ljava/lang/Object;+20
> j  org.apache.spark.sql.Dataset.collect()Ljava/lang/Object;+2
> 

Mime
View raw message