spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Agrawal <Ravi.Agra...@talentica.com>
Subject Unable to kill spark app gracefully. Unable to stop driver in cluster mode
Date Fri, 17 Jun 2016 13:11:37 GMT
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;}
--></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p><span id="ms-rterangepaste-start"></span></p>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
Hi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
While working on Spark 1.6.1, I ran into an issue with closing the Spark app.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
I tried it with deploy-mode as&nbsp;client as well as cluster:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font size="4"><span style="font-size: 14pt;"><b>Firstly,&nbsp;</b></span></font><font
size="4"><span style="font-size: 14pt;"><b>deploy-mode : client</b></span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<u>Ran the app using below command:</u></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
/usr/local/spark/spark-1.6.1-bin-hadoop2.6/bin/spark-submit --supervise --class &lt;Qualified
class name&gt; &lt;Jar path&gt;<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<u>Killed from UI by clicking the 'kill' link:</u></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
App got removed from UI but it was still running on the server.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<u>Killed using below command:</u></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
bin/spark-class org.apache.spark.deploy.Client kill spark://raviagrawal:7077 app-20160617181402-0005<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
The above command did not work. Gave below message on console:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
ClientEndpoint: Driver app-20160617181402-0005 has already finished or does not exist<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
The app was still running</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="4"><span style="font-size: 14pt;"><b>Secondly,&nbsp;</b></span></font><font
face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="4"><span style="font-size: 14pt;"><b>deploy-mode
 : cluster</b></span></font><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
&nbsp;</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><u>Ran
the app using below command:</u></span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">/usr/local/spark/spark-1.6.1-bin-hadoop2.6/bin/spark-submit
&nbsp;--master spark://raviagrawal:7077
 --deploy-mode cluster --supervise --class &lt;Qualified class name&gt; &lt;Jar
path&gt;<br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><u>Killed
from UI by clicking the 'kill' link:</u></span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">App
got removed from UI but it was still running on the server.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">Unable
to kill driver from UI</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><u>Killed
using below command:</u></span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">bin/spark-class
org.apache.spark.deploy.Client kill spark://raviagrawal:7077 driver-20160617181915-0003</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">The
above command did not work. Gave below message on console:</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">&nbsp;</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">
16/06/17 18:20:04 INFO ClientEndpoint: Kill request for driver-20160617181915-0003 submitted</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">
16/06/17 18:20:04 INFO ClientEndpoint: ... waiting before polling master for driver state</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">
16/06/17 18:20:09 INFO ClientEndpoint: ... polling master for driver state</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">
16/06/17 18:20:09 INFO ClientEndpoint: State of driver-20160617181915-0003 is RUNNING</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px;">
16/06/17 18:20:09 INFO ClientEndpoint: Driver running on 172.19.19.105:55454 (worker-20160617134955-172.19.19.105-55454)</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
&nbsp;</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">I went
through the resources available online but could not find a solution to
 the problem.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">I can
kill the process using kill -9 pid, but do not want to use it. I want to
 kill the app gracefully.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">Also
found&nbsp;<a href="https://issues.apache.org/jira/browse/SPARK-11588" target="_blank">this
 JIRA</a>&nbsp;mentioning almost the same issue, but the status of that has been
changed to Invalid as several questions were asked in it.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">Can
you please help me on this?</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">Thanks,</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">-Ravi.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 16px; margin-top:
0px; margin-bottom: 0px;">
<font face="Calibri,Arial,Helvetica,sans-serif,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe
UI Symbol,Android Emoji,EmojiSymbols" size="2"><span style="font-size: 16px;">Talentica
Software India Pvt. Ltd.</span></font></div>
<span id="ms-rterangepaste-end"></span><br>
<p></p>
</div>
</body>
</html>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message