spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yin Yang <yy201...@gmail.com>
Subject Re: Ordering two dimensional arrays of (String, Int) in the order of second element
Date Sat, 27 Feb 2016 18:46:49 GMT
Is this what you look for ?

scala> Seq((2, "a", "test"), (2, "b", "foo")).toDF("id", "a",
"b").registerTempTable("test")

scala> val df = sql("SELECT struct(id, b, a) from test")
df: org.apache.spark.sql.DataFrame = [struct(id, b, a): struct<id: int, b:
string ... 1 more field>]

scala> df.show
+----------------+
|struct(id, b, a)|
+----------------+
|      [2,test,a]|
|       [2,foo,b]|
+----------------+

You can adjust the order of the columns in struct() .

FYI

On Sat, Feb 27, 2016 at 10:25 AM, Ashok Kumar <ashok34668@yahoo.com.invalid>
wrote:

> Hello,
>
> I like to be able to solve this using arrays.
>
> I have two dimensional array of (String,Int) with 5  entries say
> arr("A",20), arr("B",13), arr("C", 18), arr("D",10), arr("E",19)
>
> I like to write a small code to order these in the order of highest Int
> column so I will have arr("A",20), arr("E",19), arr("C",18) ....
>
> What is the best way of doing this using arrays only?
>
> Thanks
>

Mime
View raw message