spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subhajit Purkayastha" <>
Subject RE: Spark 2.0 - Insert/Update to a DataFrame
Date Fri, 26 Aug 2016 20:29:26 GMT


The grains of the dataFrame are different.


I need to reduce the forecast qty (which is in the FCST DF)  based on the sales qty (coming
from the sales  order DF)


Hope it helps




From: Mike Metzger [] 
Sent: Friday, August 26, 2016 1:13 PM
To: Subhajit Purkayastha <>
Cc: user @spark <>
Subject: Re: Spark 2.0 - Insert/Update to a DataFrame


Without seeing the makeup of the Dataframes nor what your logic is for updating them, I'd
suggest doing a join of the Forecast DF with the appropriate columns from the SalesOrder DF.




On Fri, Aug 26, 2016 at 11:53 AM, Subhajit Purkayastha < <>
> wrote:

I am using spark 2.0, have 2 DataFrames, SalesOrder and Forecast. I need to update the Forecast
Dataframe record(s), based on the SaleOrder DF record. What is the best way to achieve this


View raw message