spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Subash Prabakar <subashpraba...@gmail.com>
Subject Spark2: Deciphering saving text file name
Date Tue, 09 Apr 2019 00:54:43 GMT
Hi,
While saving in Spark2 as text file - I see encoded/hash value attached in
the part files along with part number. I am curious to know what is that
value is about ?

Example:
ds.write.save(SaveMode.Overwrite).option("compression","gzip").text(path)

Produces,
part-00001-1e4c5369-6694-4012-894a-73b971fe1ab1-c000.txt.gz


1e4c5369-6694-4012-894a-73b971fe1ab1-c000 => what is this value ?

Is there any options available to remove this part or is it attached for
some reason ?

Thanks,
Subash

Mime
View raw message