spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: Using Spark on Data size larger than Memory size
Date Sat, 31 May 2014 07:14:09 GMT
Clearly thr will be impact on performance but frankly depends on what you
are trying to achieve with the dataset.

Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>



On Sat, May 31, 2014 at 11:45 AM, Vibhor Banga <vibhorbanga@gmail.com>
wrote:

> Some inputs will be really helpful.
>
> Thanks,
> -Vibhor
>
>
> On Fri, May 30, 2014 at 7:51 PM, Vibhor Banga <vibhorbanga@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I am planning to use spark with HBase, where I generate RDD by reading
>> data from HBase Table.
>>
>> I want to know that in the case when the size of HBase Table grows larger
>> than the size of RAM available in the cluster, will the application fail,
>> or will there be an impact in performance ?
>>
>> Any thoughts in this direction will be helpful and are welcome.
>>
>> Thanks,
>> -Vibhor
>>
>
>
>
> --
> Vibhor Banga
> Software Development Engineer
> Flipkart Internet Pvt. Ltd., Bangalore
>
>

Mime
View raw message