spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brandon Geise <brandonge...@gmail.com>
Subject [Spark CSV DataframeWriter] Quote options for columns on write
Date Wed, 07 Mar 2018 01:52:31 GMT
My problem is related to the need to have all records in a specific column quoted when writing
a CSV.  I assumed that by setting the options escapeQuotes to false in the options, that
fields would not have any type of quoting applied, even when that delimiter exists.  Unless
I am misunderstanding what the option actually does, it appears it doesn’t work as expected. 
Can anyone provide some information around that option?  And if the option will not do what
I’m trying to accomplish, is the best solution to set the quote option to \u0000 (My issue
here is we are sending these CSVS to a partner and I’d like it to be more in line with a
normal quoted field)?  

 

Thanks,

Brandon


Mime
View raw message