spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AJT <>
Subject Dataframe Grouping - Sorting - Mapping
Date Fri, 30 Sep 2016 10:46:36 GMT
I'm looking to do the following with my Spark dataframe
(1) val df1 = df.groupBy(<long timestamp column>)
(2) val df2 = df1.sort(<long timestamp column>)
(3) val df3 = df2.mapPartitions(<set of aggregating functions>)

I can already groupBy the column (in this case a long timestamp) - but have
no idea how then to ensure the returned GroupedData is then sorted by the
same timeStamp and the mapped to my set of functions

Appreciate any help

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message