spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subhajit Purkayastha" <spurk...@p3si.net>
Subject RE: Spark 2.0 - Insert/Update to a DataFrame
Date Fri, 26 Aug 2016 20:29:26 GMT
Mike,

 

The grains of the dataFrame are different.

 

I need to reduce the forecast qty (which is in the FCST DF)  based on the sales qty (coming
from the sales  order DF)

 

Hope it helps

 

Subhajit

 

From: Mike Metzger [mailto:mike@flexiblecreations.com] 
Sent: Friday, August 26, 2016 1:13 PM
To: Subhajit Purkayastha <spurkaya@p3si.net>
Cc: user @spark <user@spark.apache.org>
Subject: Re: Spark 2.0 - Insert/Update to a DataFrame

 

Without seeing the makeup of the Dataframes nor what your logic is for updating them, I'd
suggest doing a join of the Forecast DF with the appropriate columns from the SalesOrder DF.
 

 

Mike

 

On Fri, Aug 26, 2016 at 11:53 AM, Subhajit Purkayastha <spurkaya@p3si.net <mailto:spurkaya@p3si.net>
> wrote:

I am using spark 2.0, have 2 DataFrames, SalesOrder and Forecast. I need to update the Forecast
Dataframe record(s), based on the SaleOrder DF record. What is the best way to achieve this
functionality

 


Mime
View raw message