spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manu Zhang <>
Subject Spark driver crashed with internal error
Date Mon, 08 Apr 2019 03:00:05 GMT
Hi all,

Recently, our Spark application's (2.3.1) driver has been crashing before
exiting with the following error.

* Could not load; library not loadable; PrintAssembly is
* #*
* # A fatal error has been detected by the Java Runtime Environment:*
* #*
* #  Internal Error (sharedRuntime.cpp:834), pid=40111,
* #  fatal error: exception happened outside interpreter, nmethods and
vtable stubs at pc 0x00002ac1a832edb1*
* #*
* # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build
* # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode
linux-amd64 compressed oops)*

Similar errors were reported in older JVM and Linux  kernel versions but we are running on Java
1.8.0_131 and kernel 3.10.0-693.21.1.el7.x86_64.

Here is the stack in error report.

*Current thread (0x00002ac438005000):  JavaThread
"block-manager-slave-async-thread-pool-2" daemon [_thread_in_Java,
id=40362, stack(0x00002ac46170a000,0x00002ac46180b000)]*

*Stack: [0x00002ac46170a000,0x00002ac46180b000],  sp=0x00002ac461808cd0,
free space=1019k*
*Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
*V  []  VMError::report_and_die()+0x2ba*
*V  []  report_fatal(char const*, int, char const*)+0x59*
*V  []
SharedRuntime::continuation_for_implicit_exception(JavaThread*, unsigned
char*, SharedRuntime::ImplicitExceptionKind)+0x33a*
*V  []  JVM_handle_linux_signal+0x48a*
*V  []  signalHandler(int, siginfo*, void*)+0x43*
*C  []*
*j  java.util.concurrent.ThreadPoolExecutor$*
*v  ~StubRoutines::call_stub*
*V  []  JavaCalls::call_helper(JavaValue*, methodHandle*,
JavaCallArguments*, Thread*)+0x1056*
*V  []  JavaCalls::call_virtual(JavaValue*, KlassHandle,
Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x321*
*V  []  JavaCalls::call_virtual(JavaValue*, Handle,
KlassHandle, Symbol*, Symbol*, Thread*)+0x47*
*V  []  thread_entry(JavaThread*, Thread*)+0xa0*
*V  []  JavaThread::thread_main_inner()+0x103*
*V  []  JavaThread::run()+0x11c*
*V  []  java_start(Thread*)+0x108*
*C  []*

Has anyone seen this kind of error before ? I could provide more
information if needed.

Manu Zhang

View raw message