spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Subhash Sriram <subhash.sri...@gmail.com>
Subject Re: Run SQL on files directly
Date Sat, 08 Dec 2018 20:37:58 GMT
Hi David,

I’m not sure if that is possible, but why not just read the CSV file using the Scala API,
specifying those options, and then query it using SQL by creating a temp view?

Thanks,
Subhash 

Sent from my iPhone

> On Dec 8, 2018, at 12:39 PM, David Markovitz <Dudu.Markovitz@microsoft.com.INVALID>
wrote:
> 
> Hi
> Spark SQL supports direct querying on files (here), e.g. –
>  
> select * from csv.`/my/path/myfile.csv`
> 
>  
> Does anybody know if it possible to pass options (sep, header, encoding etc.) with this
syntax?
>  
> Thanks
>  
>  
> Best regards,
>  
> David (דודו) Markovitz
> Technology Solutions Professional, Data Platform
> Microsoft Israel
>  
> Mobile: +972-525-834-304
> Office: +972-747-119-274
>  
> <image001.png>
>  

Mime
View raw message