spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <>
Subject type inference csv dates
Date Fri, 12 Aug 2016 02:07:00 GMT
i generally like the type inference feature of the spark-sql csv
datasource, however i have been stung several times by date inference. the
problem is that when a column is converted to a date type the original data
is lost. this is not a lossless conversion. and i often have a requirement
where i need to preserve the date formatting when writing back out.

so is there a way to disable date inference all-together and treat dates as

thanks! koert

View raw message