spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Lists <lists.a...@gmail.com>
Subject Re: Reading resource files in a Spark application
Date Wed, 14 Jan 2015 16:17:55 GMT
The problem is that it gives an error message saying something to the
effect that:

URI is not hierarchical

This is consistent with your explanation.

Thanks,
arun


On Wed, Jan 14, 2015 at 1:14 AM, Sean Owen <sowen@cloudera.com> wrote:

> My hunch is that it is because the URI of a resource in a JAR file will
> necessarily be specific to where the JAR is on the local filesystem and
> that is not portable or the right way to read a resource. But you didn't
> specify the problem here.
> On Jan 14, 2015 5:15 AM, "Arun Lists" <lists.arun@gmail.com> wrote:
>
>> I experimented with using getResourceAsStream(cls, fileName) instead
>> cls.getResource(fileName).toURI. That works!
>>
>> I have no idea why the latter method does not work in Spark. Any
>> explanations would be welcome.
>>
>> Thanks,
>> arun
>>
>>
>> On Tue, Jan 13, 2015 at 6:35 PM, Arun Lists <lists.arun@gmail.com> wrote:
>>
>>> In some classes, I initialize some values from resource files using the
>>> following snippet:
>>>
>>> new File(cls.getResource(fileName).toURI)
>>>
>>> This works fine in SBT. When I run it using spark-submit, I get a bunch of errors
because the classes cannot be initialized. What can I do to make such initialization that
is Spark friendly?
>>>
>>> Thanks,
>>> arun
>>>
>>>
>>>
>>

Mime
View raw message