spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From madhu phatak <phatak....@gmail.com>
Subject Re: why generateJob is a private API?
Date Tue, 17 Mar 2015 08:56:54 GMT
Hi,
 Thank you for the response.

Regards,
Madhukara Phatak
http://datamantra.io/

On Tue, Mar 17, 2015 at 5:50 AM, Tathagata Das <tdas@databricks.com> wrote:

> It was not really meant to be pubic and overridden. Because anything you
> want to do to generate jobs from RDDs can be done using DStream.foreachRDD
>
> On Sun, Mar 15, 2015 at 11:14 PM, madhu phatak <phatak.dev@gmail.com>
> wrote:
>
>> Hi,
>>  I am trying to create a simple subclass of DStream.  If I understand
>> correctly, I should override *compute *lazy operations and *generateJob*
>> for actions. But when I try to override, generateJob it gives error saying
>> method is private to the streaming package. Is my approach is correct or am
>> I  missing something?
>>
>>
>> Regards,
>> Madhukara Phatak
>> http://datamantra.io/
>>
>
>

Mime
View raw message