spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: calculate time difference between consecutive rows
Date Thu, 21 Jul 2016 07:19:10 GMT
Hi,

What was the code you tried? You should use the built-in window
aggregates (windows) functions or create one yourself. I haven't tried
lag before (and don't think it's what you need really).

Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski


On Thu, Jul 21, 2016 at 6:27 AM, Divya Gehlot <divya.htconex@gmail.com> wrote:
> I have a dataset of time as shown below :
> Time1
> 07:30:23
> 07:34:34
> 07:38:23
> 07:39:12
> 07:45:20
>
> I need to find the diff between two consecutive rows
> I googled and found the lag function in spark helps in finding it .
> but its  giving me null in the result set.
>
> Would really appreciate the help.
>
>
> Thanks,
> Divya
>

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message