spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakob Odersky <ja...@odersky.com>
Subject Re: Issue with wholeTextFiles
Date Tue, 22 Mar 2016 22:20:02 GMT
Are you sure the file name starts with "hdfs://"? I can't say for sure, but
the first character of the blacked-out text looks like a sloped line. In
that case, I would assume its a slash and thus referring to a local file.

On Tue, Mar 22, 2016 at 2:06 AM, Sarath Chandra <
sarathchandra.josyam@algofusiontech.com> wrote:

> It is pointing to HDFS file.
>
> It works well from spark shell but when try same thing through a program
> using spark context it fails.
>
> On Tue, Mar 22, 2016 at 2:32 PM, Akhil Das <akhil@sigmoidanalytics.com>
> wrote:
>
>> Is it pointing to an HDFS file or a local file?
>>
>> Thanks
>> Best Regards
>>
>> On Tue, Mar 22, 2016 at 1:26 PM, Sarath Chandra <
>> sarathchandra.josyam@algofusiontech.com> wrote:
>>
>>> Hi Akhil,
>>>
>>> Please find below the stack trace.
>>>
>>>
>>> ‚Äč
>>> ~Sarath
>>>
>>>
>>>
>>> On Tue, Mar 22, 2016 at 12:28 PM, Akhil Das <akhil@sigmoidanalytics.com>
>>> wrote:
>>>
>>>> Can you paste the exception stack here?
>>>>
>>>> Thanks
>>>> Best Regards
>>>>
>>>> On Mon, Mar 21, 2016 at 1:42 PM, Sarath Chandra <
>>>> sarathchandra.josyam@algofusiontech.com> wrote:
>>>>
>>>>> I'm using Hadoop 1.0.4 and Spark 1.2.0.
>>>>>
>>>>> I'm facing a strange issue. I have a requirement to read a small file
>>>>> from HDFS and all it's content has to be read at one shot. So I'm using
>>>>> spark context's wholeTextFiles API passing the HDFS URL for the file.
>>>>>
>>>>> When I try this from a spark shell it's works as mentioned in the
>>>>> documentation, but when I try the same through program (by submitting
job
>>>>> to cluster) I get FileNotFoundException. I have all compatible JARs in
>>>>> place.
>>>>>
>>>>> Please help.
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message