spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine DUBOIS <>
Subject Spark and ZStandard
Date Mon, 19 Aug 2019 15:38:43 GMT
I'm using hadoop 3.1.2 with Yarn and Spark 2.4.2: 
I'm trying to read file compressed with zstd command line from spark shell. 
However after a huge fight to finally understand issue in library import and other stuff,
I no longer face error when trying to read those files. 
However If I try to read a simple zstd file in spark-shell I open read it but the output is
scramble with unprintable caracters all over the place and the file is not really readable
at all. 
Is there any way to read zstd file generated from official ZStandard tool in Spark ? or any
way to generate readable zstd file to spark ? 

Best regards 


View raw message