spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <mich.talebza...@gmail.com>
Subject Re: Sorting the dataframe
Date Fri, 04 Mar 2016 08:32:18 GMT
Try this example, similar to yours. DF should sufficient

val a = Seq(("Mich",20), ("Christian", 18), ("James",13), ("Richard",16))
// Sort option 1 using tempTable
val b = a.toDF("Name","score").registerTempTable("tmp")
sql("select Name,score from tmp order by score desc").show
// Sort option 2 with FP
a.toDF("Name","score").sort(desc("score")).show
sys.exit

HTH


Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



On 4 March 2016 at 08:18, Angel Angel <areyouangel90@gmail.com> wrote:

> hello sir,
>
>  i want to sort the following table as per the *count*
>
> value count
> 52639 22
> 75243 4
> 13 55
> 56 5
> 185463 45
> 324364 32
>
>
> So first i convert the my dataframe to to rdd to sort the table.
>
> val k = table.rdd
>
> convert the rdd array into key value pair.
>
> val s =k.take(6)
>
> val rdd = s.map(x=> x(1),(x(0)).
> rdd.sortByKey
>
>
>
> this is my all operations i did to sort the table.
>
> Please can you suggest me the better way to sort the table
>

Mime
View raw message