spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Preece" <tepre...@mail.com>
Subject Intermittent timeout failure org/apache/spark/sql/hive/thriftserver/CliSuite.scala
Date Wed, 12 Aug 2015 08:30:31 GMT
<html><head></head><body><div style="font-family: Verdana;font-size:
12.0px;"><div><span style="color: rgb(34, 34, 34); font-family: arial, sans-serif;
font-size: 12.8000001907349px; line-height: normal;">I was just debugging an intermittent
timeout failure in the testsuite CliSuite.scala</span></div>

<div>
<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">&nbsp;</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">I traced it down to a timing window in the Scala library class sys.process.ProcessImpl.scala.
Sometimes the input pipe to a process becomes &#39;None&#39; before the process has
had a chance to read any input at all. This results in a subsequent timeout failure in CliSuite.</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">&nbsp;</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">There is a JIRA on the Scala side&nbsp;<a href="https://issues.scala-lang.org/browse/SI-8768"
style="color: rgb(17, 85, 204);" target="_blank">https://issues.scala-lang.org/<wbr/>browse/SI-8768</a>&nbsp;which
should resolve the problem ( although that JIRA was raised for slightly different symptoms
) and this is fixed in scala 2.12.0</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">&nbsp;</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">I also wonder if some fixes ( e.g <a href="https://issues.apache.org/jira/browse/SPARK-7973"
style="color: rgb(17, 85, 204);" target="_blank">https://issues.apache.org/<wbr/>jira/browse/SPARK-7973</a>&nbsp;)
may be a result of this Scala issue.</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">&nbsp;</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">I am new to the Spark community. Is there a preferred way to track
the fact the Spark testcase CliSuite has a dependency on the above Scala issue ?</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">&nbsp;</div>

<div style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 12.8000001907349px;
line-height: normal;">Tim Preece</div>
</div></div></body></html>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message