spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siavash Namvar <sns...@gmail.com>
Subject How can I use pyspark to upsert one row without replacing entire table
Date Wed, 12 Aug 2020 13:18:41 GMT
Hi,
<https://stackoverflow.com/posts/63377211/timeline>

I have a use case, and read data from a db table and need to update few
rows based on primary key without replacing the entire table.

for instance if I have 3 following rows

-------------------
id | fname         -------------------
 1 | john          -------------------
 2 | Steve         -------------------
 3 | Jack         -------------------

And I would like to update the row with id=2 from Steve to Michael without
replacing the entire table and the outpur looks like

-------------------
id | fname         -------------------
 1 | john          -------------------
 2 | Michael         -------------------
 3 | Jack         -------------------

Keep in mind the actual db table is so huge and database is old and cannot
read and replace entire table

Thanks

Mime
View raw message