spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aurélien Mazoyer <aurel...@aepsilon.com>
Subject Detailed information about corrupted record
Date Thu, 09 Sep 2021 17:10:31 GMT
Hello,

I use a DataFrameReader and I use permissive mode for corrupted records. I
would like to have more information about corrupted records: for example,
in the case of a schema mismatch, something like: "Invalid type. Expected
Integer but got String for column number 12 (in case of csv).
By the way, I am unfortunately stuck with spark 2 :-).

Thank you for your advice,

Aurelien

Mime
View raw message