spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fang, Mike" <chuf...@paypal.com.INVALID>
Subject control the number of reducers for groupby in data frame
Date Wed, 05 Aug 2015 05:47:43 GMT
Hi,

Does anyone know how I could control the number of reducer when we do operation such as groupie
For data frame?
I could set spark.sql.shuffle.partitions in sql but not sure how to do in df.groupBy("XX")
api.

Thanks,
Mike

Mime
View raw message