spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ameet Kini <ameetk...@gmail.com>
Subject Re: why is NextIterator private?
Date Fri, 21 Feb 2014 19:35:05 GMT
I'm looking at NewHadoopRDD and looks like there is no equivalent of
NextIterator there. I'm tempted to go with extending that.

Interestingly, in NewHadoopRDD, there is no concept of caching the
configuration if one has not been broadcasted, whereas in HadoopRDD, there
is a cache. I'm now tempted to go with extending NewHadoopRDD, and would
appreciate some insight from the developers on what was the thinking behind
keeping that out.

Thanks,
Ameet


On Fri, Feb 21, 2014 at 2:26 PM, Ameet Kini <ameetkini@gmail.com> wrote:

> On that same token, HadoopPartition.
>
> Yes, I can copy them verbatim and change them as they are small enough.
> But sub-classing would have been nicer.
>
> Ameet
>
>
> On Fri, Feb 21, 2014 at 2:16 PM, Ameet Kini <ameetkini@gmail.com> wrote:
>
>>
>> I'm looking to subclass HadoopRDD and was hoping to subclass NextIterator
>> in compute().
>>
>> Thanks,
>> Ameet
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message