spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serega Sheypak <>
Subject Append more files to existing partitioned data
Date Sat, 17 Mar 2018 12:18:52 GMT
Hi, I', using spark-sql to process my data and store result as parquet
partitioned by several columns

  .partitionBy("year", "month", "day", "hour", "workflowId")

I want to run more jobs that will produce new partitions or add more files
to existing partitions.
What is the right way to do it?

View raw message