spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Tanase <atan...@adobe.com>
Subject Re: Accumulator of rows?
Date Fri, 02 Oct 2015 11:23:44 GMT
Have you seen window functions?
https://databricks.com/blog/2015/07/15/introducing-window-functions-in-spark-sql.html

From: "Saif.A.Ellafi@wellsfargo.com<mailto:Saif.A.Ellafi@wellsfargo.com>"
Date: Thursday, October 1, 2015 at 9:44 PM
To: "user@spark.apache.org<mailto:user@spark.apache.org>"
Subject: Accumulator of rows?

Hi all,

I need to repeat a couple rows from a dataframe by n times each. To do so, I plan to create
a new Data Frame, but I am being unable to find a way to accumulate “Rows” somewhere,
as this might get huge, I can’t accumulate into a mutable Array, I think?.

Thanks,
Saif

Mime
View raw message