spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emre Sevinc <emre.sev...@gmail.com>
Subject Re: Tuning Spark Streaming jobs
Date Mon, 22 Dec 2014 17:42:41 GMT
Great guide!

Two typos:

  - "... are an esse part ...": it must be "... are an essential part ..."

  - "... express that constrain in Mesos": it must be "express that
constraint in Mesos"


--
Emre Sevinç
https://be.linkedin.com/in/emresevinc


On Mon, Dec 22, 2014 at 5:33 PM, Gerard Maas <gerard.maas@gmail.com> wrote:
>
> Hi,
>
> After facing issues with the performance of some of our Spark Streaming
>  jobs, we invested quite some effort figuring out the factors that affect
> the performance characteristics of a Streaming job. We  defined an
> empirical model that helps us reason about Streaming jobs and applied it to
> tune the jobs in order to maximize throughput.
>
> We have summarized our findings in a blog post with the intention of
> collecting feedback and hoping that it is useful to other Spark Streaming
> users facing similar issues.
>
>  http://www.virdata.com/tuning-spark/
>
> Your feedback is welcome.
>
> With kind regards,
>
> Gerard.
> Data Processing Team Lead
> Virdata.com
> @maasg
>
>
>

-- 
Emre Sevinc

Mime
View raw message