spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "陈宇航" <yuhang.c...@foxmail.com>
Subject 回复:SparkLauncher is blocked until main process is killed.
Date Fri, 30 Oct 2015 02:58:55 GMT
Here's a part of the jstack output.

The release is 1.5.1.

---原始邮件---
发件人: "Ted Yu"<yuzhihong@gmail.com>
发送时间: 2015年10月30日 10:11:34
收件人: "jey"<jey@cs.berkeley.edu>;
抄送: "陈宇航"<yuhang.chen@foxmail.com>;"user"<user@spark.apache.org>;
主题: Re: SparkLauncher is blocked until main process is killed.


Which Spark release are you using ?

Please note the typo in email subject (corrected as of this reply)

On Thu, Oct 29, 2015 at 7:00 PM, Jey Kottalam <jey@cs.berkeley.edu> wrote:
Could you please provide the jstack output? That would help the devs identify the blocking
operation more easily.


On Thu, Oct 29, 2015 at 6:54 PM, 陈宇航 <yuhang.chen@foxmail.com> wrote:
I tried to use SparkLauncher (org.apache.spark.launcher.SparkLauncher) to submit a Spark Streaming
job, however, in my test, the SparkSubmit process got stuck in the "addJar" procedure. Only
when the main process (the caller of SparkLauncher) is killed, the submit procedeure continues
to run. I ran jstack for the process, it seems jetty was blocking it, and I'm pretty sure
there was no port conflicts.


The environment is RHEL(RedHot Enterprise Linux) 6u3 x64, Spark runs in standalone mode.


Did this happen to any of you?
Mime
View raw message