spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From raja kbv <>
Subject Re: How to ignore case in dataframe groupby?
Date Wed, 30 Dec 2015 17:05:21 GMT
Solutions from Eran & Yanbo are working well. Thank you. 
Your solution worked with a small change. DF.withColumn("upper-code",upper(df("countrycode"))).

This creates a new column "upper-code". Is there a way to update the column or create a new
df with update column? 


    On Thursday, 24 December 2015 6:17 PM, Eran Witkon <> wrote:

 Use DF.withColumn("upper-code",df("countrycode).toUpper))or just run a map function that
does the same
On Thu, Dec 24, 2015 at 2:05 PM Bharathi Raja <> wrote:

Values in a dataframe column named countrycode are in different cases. Eg: (US, us).  groupBy
& count gives two rows but the requirement is to ignore case for this operation.
1) Is there a way to ignore case in groupBy? Or
2) Is there a way to update the dataframe column countrycode to uppercase?

Thanks in advance.


View raw message