spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From St├ęphane Verlet <kaweahsoluti...@gmail.com>
Subject Re: Spark job taking 10s to allocate executors and memory before submitting job
Date Thu, 28 Sep 2017 14:18:15 GMT
Sounds like such a small job , if you running in on a cluster  have you
consider simply running it locally  (master = local) ?

On Wed, Sep 27, 2017 at 7:06 AM, navneet sharma <navneetsharma0505@gmail.com
> wrote:

> Hi,
>
> I am running spark job taking total 18s, in that 8 seconds for actual
> processing logic(business logic) and 10s for allocating executors and
> memory. How to reduce initial time.
>
> Any ideas how to reduce time before spark job goes to submit state.
>
> thanks,
>
> Navneet Sharma
>

Mime
View raw message