spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Read a HDFS file from Spark using HDFS API
Date Fri, 14 Nov 2014 15:51:43 GMT
[image: Inline image 1]


Thanks
Best Regards

On Fri, Nov 14, 2014 at 9:18 PM, Bui, Tri <
Tri.Bui@verizonwireless.com.invalid> wrote:

> It should be
>
>
>
> val file = sc.textFile("hdfs:///localhost:9000/sigmoid/input.txt")
>
>
>
> 3 “///”
>
>
>
> Thanks
>
> Tri
>
>
>
> *From:* rapelly kartheek [mailto:kartheek.mbms@gmail.com]
> *Sent:* Friday, November 14, 2014 9:42 AM
> *To:* Akhil Das; user@spark.apache.org
> *Subject:* Re: Read a HDFS file from Spark using HDFS API
>
>
>
> No. I am not accessing hdfs from either shell or a spark application. I
> want to access from spark "Scheduler code".
>
>
>
> I face an error when I use sc.textFile() as SparkContext wouldn't have
> been created yet. So, error says: "sc not found".
>
>
>
> On Fri, Nov 14, 2014 at 9:07 PM, Akhil Das <akhil@sigmoidanalytics.com>
> wrote:
>
> like this?
>
>
>
> val file = sc.textFile("hdfs://localhost:9000/sigmoid/input.txt")
>
>
> Thanks
>
> Best Regards
>
>
>
> On Fri, Nov 14, 2014 at 9:02 PM, rapelly kartheek <kartheek.mbms@gmail.com>
> wrote:
>
> Hi,
>
> I am trying to read a HDFS file from Spark "scheduler code". I could find
> how to write hdfs read/writes in java.
>
>
>
> But I  need to access hdfs from spark using scala. Can someone please help
> me in this regard.
>
>
>
>
>

Mime
View raw message