spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: wierd caching
Date Sun, 09 Nov 2014 06:35:26 GMT
It might mean that some partition was computed on two nodes, because a task for it wasn't able
to be scheduled locally on the first node. Did the RDD really have 426 partitions total? You
can click on it and see where there are copies of each one.

Matei

> On Nov 8, 2014, at 10:16 PM, Nathan Kronenfeld <nkronenfeld@oculusinfo.com> wrote:
> 
> RDD Name	Storage Level	Cached Partitions	Fraction Cached	Size in Memory	Size in Tachyon
Size on Disk
> 8 <http://hadoop-s1.oculus.guest:4042/storage/rdd?id=8>	Memory Deserialized 1x
Replicated	426	107%	59.7 GB	0.0 B	0.0 B
> Anyone understand what it means to have more than 100% of an rdd cached?
> 
> Thanks,
>                 -Nathan
> 


Mime
View raw message