spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vvshvv <vvs...@gmail.com>
Subject Re:Docker image for Spark streaming app
Date Sun, 08 Jan 2017 21:02:27 GMT
<div dir="ltr">Hi,<br>
</div><div dir="ltr"><br>
</div><div dir="ltr">I am running spark streaming job using spark jobserver via
this image:<br>
</div><div dir="ltr"><br>
</div><div dir="ltr"><a href="https://hub.docker.com/r/depend/spark-jobserver">https://hub.docker.com/r/depend/spark-jobserver</a>/.<br>
</div><div dir="ltr"><br>
</div><div dir="ltr">It works well in standalone (using mesos job does not make
progress). Spark jobserver that supports Spark 2.0 has new API that is only suitable for non-streaming
jobs as for streaming you have to specify StreamingContextFactory that still uses old API,
so you can just use old API for now.<br>
</div><div dir="ltr"><br>
</div><div dir="ltr"><br>
</div><div dir="ltr"><br>
</div><div class="wps_signature">Sent from my Mi phone</div><div class="wps_quotion">On
shyla deshpande &lt;deshpandeshyla@gmail.com&gt;, Jan 8, 2017 11:51 PM wrote:<br
type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex"><p></p><div dir="ltr">I looking for a docker
image that I can use from docker hub for running a spark streaming app with scala and spark
2.0 +.<div><br></div><div>I am new to docker and unable to find one
image from docker hub that suits my needs. Please let me know if anyone is using a docker
for spark streaming app and share your experience.</div><div><br></div><div>Thanks</div></div>
</blockquote></div>
Mime
View raw message