spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jem Tucker <jem.tuc...@gmail.com>
Subject Re: Custom Partitioner
Date Tue, 01 Sep 2015 09:52:12 GMT
Hi,

You just need to extend Partitioner and override the numPartitions and
getPartition methods, see below

class MyPartitioner extends partitioner {
  def numPartitions: Int = // Return the number of partitions
  def getPartition(key Any): Int = // Return the partition for a given key
}

On Tue, Sep 1, 2015 at 10:15 AM shahid qadri <shahidashraff@icloud.com>
wrote:

> Hi Sparkians
>
> How can we create a customer partition in pyspark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message