spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerard Maas <gerard.m...@gmail.com>
Subject Tuning Spark Streaming jobs
Date Mon, 22 Dec 2014 16:33:17 GMT
Hi,

After facing issues with the performance of some of our Spark Streaming
 jobs, we invested quite some effort figuring out the factors that affect
the performance characteristics of a Streaming job. We  defined an
empirical model that helps us reason about Streaming jobs and applied it to
tune the jobs in order to maximize throughput.

We have summarized our findings in a blog post with the intention of
collecting feedback and hoping that it is useful to other Spark Streaming
users facing similar issues.

 http://www.virdata.com/tuning-spark/

Your feedback is welcome.

With kind regards,

Gerard.
Data Processing Team Lead
Virdata.com
@maasg

Mime
View raw message