spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajay Chander <itsche...@gmail.com>
Subject Re: Converting a string of format of 'dd/MM/yyyy' in Spark sql
Date Thu, 24 Mar 2016 14:23:39 GMT
Mich,

Can you try the value for paymentdata to this
format  paymentdata='2015-01-01 23:59:59' , to_date(paymentdate) and see if
it helps.

On Thursday, March 24, 2016, Tamas Szuromi <tamas.szuromi@odigeo.com.invalid>
wrote:

> Hi Mich,
>
> Take a look
> https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/sql/functions.html#unix_timestamp(org.apache.spark.sql.Column,%20java.lang.String)
>
> cheers,
> Tamas
>
>
> On 24 March 2016 at 14:29, Mich Talebzadeh <mich.talebzadeh@gmail.com
> <javascript:_e(%7B%7D,'cvml','mich.talebzadeh@gmail.com');>> wrote:
>
>>
>> Hi,
>>
>> I am trying to convert a date in Spark temporary table
>>
>> Tried few approaches.
>>
>> scala> sql("select paymentdate, to_date(paymentdate) from tmp")
>> res21: org.apache.spark.sql.DataFrame = [paymentdate: string, _c1: date]
>>
>>
>> scala> sql("select paymentdate, to_date(paymentdate) from tmp").first
>> *res22: org.apache.spark.sql.Row = [10/02/2014,null]*
>>
>> My date is stored as String dd/MM/yyyy as shown above. However, to_date()
>> returns null!
>>
>>
>> Thanks
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>>
>
>

Mime
View raw message