spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "deenar.toraskar" <deenar.toras...@db.com>
Subject Re: Equally weighted partitions in Spark
Date Fri, 02 May 2014 06:14:33 GMT
Yes

On a job I am currently running, 99% of the partitions finish within seconds
and a couple of partitions take around and hour to finish. I am pricing some
instruments and complex instruments take far longer to price than plain
vanilla ones. If I could distribute these complex instruments evenly, the
overall job times would greatly reduce.


Deenar



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Equally-weighted-partitions-in-Spark-tp5171p5208.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message