spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andras Nemeth <andras.nem...@lynxanalytics.com>
Subject Re: Storage information about an RDD from the API
Date Tue, 06 May 2014 08:36:22 GMT
Thanks Koert, very useful!


On Tue, Apr 29, 2014 at 6:41 PM, Koert Kuipers <koert@tresata.com> wrote:

> SparkContext.getRDDStorageInfo
>
>
> On Tue, Apr 29, 2014 at 12:34 PM, Andras Nemeth <
> andras.nemeth@lynxanalytics.com> wrote:
>
>> Hi,
>>
>> Is it possible to know from code about an RDD if it is cached, and more
>> precisely, how many of its partitions are cached in memory and how many are
>> cached on disk? I know I can get the storage level, but I also want to know
>> the current actual caching status. Knowing memory consumption would also be
>> awesome. :)
>>
>> Basically what I'm looking for is the information on the storage tab of
>> the UI, but accessible from the API.
>>
>> Thanks,
>> Andras
>>
>
>

Mime
View raw message