spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "1427357147@qq.com" <1427357...@qq.com>
Subject the meaining of "samplePointsPerPartitionHint" in RangePartitioner
Date Tue, 20 Mar 2018 07:59:29 GMT
HI  all,

The belowing is the code of RangePartitioner.
class RangePartitioner[K : Ordering : ClassTag, V](
    partitions: Int,
    rdd: RDD[_ <: Product2[K, V]],
    private var ascending: Boolean = true,
    val samplePointsPerPartitionHint: Int = 20)
I feel puzzled about the samplePointsPerPartitionHint.
My issue is :
    what is the samplePointsPerPartitionHint used for please?
If I set samplePointsPerPartitionHint as 1000000 or 20,what will happed please?

Thanks.

Robin Shao




1427357147@qq.com
Mime
View raw message