spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davies Liu <dav...@databricks.com>
Subject Re: DataFrame equivalent to RDD.partionByKey
Date Tue, 09 Aug 2016 22:42:13 GMT
I think you are looking for `def repartition(numPartitions: Int,
partitionExprs: Column*)`

On Tue, Aug 9, 2016 at 9:36 AM, Stephen Fletcher
<stephen.fletcher@gmail.com> wrote:
> Is there a DataFrameReader equivalent to the RDD's partitionByKey for RDD?
> I'm reading data from a file data source and I want to partition this data
> I'm reading in to be partitioned the same way as the data I'm processing
> through a spark streaming RDD in the process.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message