Hi,
I have Spark streaming app(1m batch) writing parquet data to a partition e.g. 
val hdfsPath = s"$dbPath/$tableName/year=$year/month=$month/day=$day"
df.write.mode(SaveMode.Append).parquet(hdfsPath)
I wonder would I lose data if I overwrite this partition with Hive (compaction/deduplication) while Spark is adding more data to it every minute. (hive query can take > 2 minutes)

Thanks,
Artur Sukhenko