spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vadim Semenov <>
Subject Re: How to avoid unnecessary spark starkups on every request?
Date Wed, 02 Nov 2016 14:45:28 GMT
Take a look at or

you can launch a persistent spark context and then submit your jobs using a
already running context

On Wed, Nov 2, 2016 at 3:34 AM, Fanjin Zeng <>

>  Hi,
>  I am working on a project that takes requests from HTTP server and
> computes accordingly on spark. But the problem is when I receive many
> request at the same time, users have to waste a lot of time on the
> unnecessary startups that occur on each request. Does Spark have built-in
> job scheduler function to solve this problem or is there any trick can be
> used to avoid these unnecessary startups?
>  Thanks,
>  Fanjin
> ---------------------------------------------------------------------
> To unsubscribe e-mail:

View raw message