spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selvam Raman <sel...@gmail.com>
Subject Update DF record with delta data in spark
Date Sun, 02 Apr 2017 13:57:35 GMT
Hi,

Table 1:(old File)

name number  salray
Test1 1 10000
Test2 2 10000
Table 2: (Delta File)

name    number  salray
Test1 1 40000
Test3 3 20000


​i do not have date stamp field in this table. Having composite key of name
and number fields.

Expected Result

name number  salray
Test1 1 40000
Test2 2 10000
Test3 3 20000


Current approach:

1) Delete row in table1 where table1.composite key = table2.composite key.
2) Union all table and table2 to get updated result.


is this right approach?. is there any other way to achieve it?​

-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Mime
View raw message