spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Reading resource files in a Spark application
Date Wed, 14 Jan 2015 09:14:41 GMT
My hunch is that it is because the URI of a resource in a JAR file will
necessarily be specific to where the JAR is on the local filesystem and
that is not portable or the right way to read a resource. But you didn't
specify the problem here.
On Jan 14, 2015 5:15 AM, "Arun Lists" <lists.arun@gmail.com> wrote:

> I experimented with using getResourceAsStream(cls, fileName) instead
> cls.getResource(fileName).toURI. That works!
>
> I have no idea why the latter method does not work in Spark. Any
> explanations would be welcome.
>
> Thanks,
> arun
>
>
> On Tue, Jan 13, 2015 at 6:35 PM, Arun Lists <lists.arun@gmail.com> wrote:
>
>> In some classes, I initialize some values from resource files using the
>> following snippet:
>>
>> new File(cls.getResource(fileName).toURI)
>>
>> This works fine in SBT. When I run it using spark-submit, I get a bunch of errors
because the classes cannot be initialized. What can I do to make such initialization that
is Spark friendly?
>>
>> Thanks,
>> arun
>>
>>
>>
>

Mime
View raw message