On Sat, Oct 19, 2013 at 9:11 PM, Mark Hamstra <> wrote:
What you are looking for is 'deps':

abstract class RDD[T: ClassManifest](
    @transient private var sc: SparkContext,
    @transient private var deps: Seq[Dependency[_]]
  ) extends Serializable with Logging {...}

On Sat, Oct 19, 2013 at 5:53 PM, dachuan <> wrote:
Hi, all,

I am pretty new to spark internal world, and currently I am interested in the fault tolerance in spark.

a little surprised, i didn't find "lineage" keyword in directory "incubator-spark/core/src/main/scala/org/apache/spark", I thought it must be there because the nsdi 2012 paper mentioned it.

[huangda@dachuan-desktop spark]$ find -type f | xargs grep -in "lineage"
[huangda@dachuan-desktop spark]$

Could you please point me into the right source code file if you happen to know?

thank you,

Dachuan Huang
Cellphone: 614-390-7234
2015 Neil Avenue
Ohio State University
Columbus, Ohio