spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From smartzjp <>
Subject Re: Writing Spark SQL output in Local and HDFS path
Date Thu, 19 Jan 2017 11:05:49 GMT
Beacause the reduce number will be not one, so it will out put a fold on the HDFS,  You can
use  “result.write.csv(foldPath)”.


  Can anyone please let us know how to write the output of the Spark SQL  in
Local  and HDFS path using Scala code.

Code :-

scala>  val result = sqlContext.sql("select empno , name from emp");
scala >;

If I give the command then It will print the output in the console.
I need to redirect the output in local file as well as HDFS file.
with the delimiter as "|".

We tried with the below code
 result.saveAsTextFile ("home/Prasad/result.txt")
It is not working as expected.

Prasad. T

View raw message