spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: SPARK PERFORMANCE TUNING
Date Wed, 21 Sep 2016 12:14:42 GMT
Do you mind sharing what your software does? What is the input data size? What is the spark
version and apis used? How many nodes? What is the input data format? Is compression used?

> On 21 Sep 2016, at 13:37, Trinadh Kaja <ktr.hadoop13@gmail.com> wrote:
> 
> Hi all,
> 
> how to increase spark performance ,i am using pyspark.
> 
> cluster info :
> 
> Total memory :600gb
> Cores            :96
> 
> command :
> spark-submit --master  yarn-client --executor-memory 10G --num-executors 50 --executor-cores
2 --driver-memory 10g --queue thequeue
>  
> 
> please help on this 
> 
> -- 
> Thanks&Regards
> K.Trinadh
> Ph-7348826118

Mime
View raw message