spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anil Kulkarni <anil...@gmail.com>
Subject Re: Spark CSV Quote only NOT NULL
Date Thu, 11 Jul 2019 23:09:49 GMT
Hi Swetha,

Thank you.
But we need the data to be quoted with ".
and when a field is null, we dont need the quotes around it.

Example:
"A",,"B","C"
Thanks
Anil

On Thu, Jul 11, 2019, 1:51 PM Swetha Ramaiah <swetha.ramaiah@gmail.com>
wrote:

> If you are using Spark 2.4.0, I think you can try something like this:
>
> .option("quote", "\u0000")
> .option("emptyValue", “”)
>
> .option("nullValue", null)
>
> Regards
> Swetha
>
>
>
> On Jul 11, 2019, at 1:45 PM, Anil Kulkarni <anil77k@gmail.com> wrote:
>
> Hi Spark users,
>
> My question is :
> I am writing a Dataframe to csv.
> Option i am using as
> .option("quoteAll","true").
>
> This is quoting even null values and making them appear as an empty
> string.
>
> How do i make sure that quotes are enabled only for non null values?
>
> --
> Cheers,
> Anil Kulkarni
> about.me/anilkulkarni
> [image: Anil Kulkarni on about.me]
>  http://anilkulkarni.com/ <http://about.me/anilkulkarni>
>
>
>

Mime
View raw message