spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From smartzjp <zjp_j...@163.com>
Subject Re: Writing Spark SQL output in Local and HDFS path
Date Thu, 19 Jan 2017 11:44:11 GMT

It’s since spark version 2.0.0,  if you are using under the version, you can try the below
code.

result.write.format("csv").save(path)


--------------

Hi,
  I tried the below code, as 
result.write.csv(home/Prasad/)   
It is not working,
  It says

Error: value csv is not member of org.apache.spark.sql.DataFrameWriter.

Regards
Prasad


On Thu, Jan 19, 2017 at 4:35 PM, smartzjp <zjp_jdev@163.com> wrote:
Beacause the reduce number will be not one, so it will out put a fold on the HDFS,  You can
use  “result.write.csv(foldPath)”.



--------------

Hi,
  Can anyone please let us know how to write the output of the Spark SQL  in
Local  and HDFS path using Scala code.

Code :-

scala>  val result = sqlContext.sql("select empno , name from emp");
scala > result.show();

If I give the command result.show() then It will print the output in the console.
I need to redirect the output in local file as well as HDFS file.
with the delimiter as "|".

We tried with the below code
 result.saveAsTextFile ("home/Prasad/result.txt")
It is not working as expected.


-- 
----------------------------------------------
Prasad. T



-- 
----------------------------------------------
Regards,
RAVI PRASAD. T


Mime
View raw message