spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naresh Peshwe <nareshpeshwe12...@gmail.com>
Subject Option for silent failure while reading a list of files.
Date Mon, 01 Jul 2019 01:52:20 GMT
Hi All,
When I try to read a list parquet files from S3, my application errors out
if even one of the files are absent. When I searched for solutions most of
them suggested filtering the list of files (on presence) before calling
read.
Shouldn't this be handled by Spark by providing an option for continuing
without throwing an error? If not, could you point me to the thread where
this was discussed upon.


Regards,
Naresh

Mime
View raw message