spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zalzberg, Idan (Agoda)" <>
Subject RE: unsafe memory access in spark 1.2.1
Date Sun, 01 Mar 2015 16:32:15 GMT
My run time version is:

java version "1.7.0_75"
OpenJDK Runtime Environment (rhel- u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)


From: Ted Yu []
Sent: Sunday, March 01, 2015 10:18 PM
To: Zalzberg, Idan (Agoda)
Subject: Re: unsafe memory access in spark 1.2.1

What Java version are you using ?


On Sun, Mar 1, 2015 at 7:03 AM, Zalzberg, Idan (Agoda) <<>>
I am using spark 1.2.1, sometimes I get these errors sporadically:
Any thought on what could be the cause?


2015-02-27 15:08:47 ERROR SparkUncaughtExceptionHandler:96 - Uncaught exception in thread
Thread[Executor task launch worker-25,5,main]
java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled
Java code
                    at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62)
                    at org.apache.spark.serializer.DeserializationStream$$anon$1.getNext(Serializer.scala:133)
                    at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:71)
                    at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:350)
                    at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)
                    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
                    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
                    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
                    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
                    at org.apache.spark.util.collection.ExternalSorter.spillToPartitionFiles(ExternalSorter.scala:365)
                    at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:211)
                    at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63)
                    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
                    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
                    at org.apache.spark.executor.Executor$
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(
                    at java.util.concurrent.ThreadPoolExecutor$

This message is confidential and is for the sole use of the intended recipient(s). It may
also be privileged or otherwise protected by copyright or other legal rules. If you have received
it by mistake please let us know by reply email and delete it from your system. It is prohibited
to copy this message or disclose its content to anyone. Any confidentiality or privilege is
not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All
messages sent to and from Agoda may be monitored to ensure compliance with company policies,
to protect the company's interests and to remove potential malware. Electronic messages may
be intercepted, amended, lost or deleted, or contain viruses.

View raw message