spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priya Ch <learnings.chitt...@gmail.com>
Subject Re: passing SparkContext as parameter
Date Mon, 21 Sep 2015 11:27:10 GMT
can i use this sparkContext on executors ??
In my application, i have scenario of reading from db for certain records
in rdd. Hence I need sparkContext to read from DB (cassandra in our case),

If sparkContext couldn't be sent to executors , what is the workaround for
this ??????

On Mon, Sep 21, 2015 at 3:06 PM, Petr Novak <oss.mlists@gmail.com> wrote:

> add @transient?
>
> On Mon, Sep 21, 2015 at 11:27 AM, Priya Ch <learnings.chitturi@gmail.com>
> wrote:
>
>> Hello All,
>>
>>     How can i pass sparkContext as a parameter to a method in an object.
>> Because passing sparkContext is giving me TaskNotSerializable Exception.
>>
>> How can i achieve this ?
>>
>> Thanks,
>> Padma Ch
>>
>
>

Mime
View raw message