spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Beabes <mailinglist...@gmail.com>
Subject Naming files while saving a Dataframe
Date Fri, 16 Jul 2021 23:10:14 GMT
We've two (or more) jobs that write data into the same directory via a
Dataframe.save method. We need to be able to figure out which job wrote
which file. Maybe provide a 'prefix' to the file names. I was wondering if
there's any 'option' that allows us to do this. Googling didn't come up
with any solution so thought of asking the Spark experts on this mailing
list.

Thanks in advance.

Mime
View raw message