spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From harsh2005_7 <>
Subject Re: Spark RDD member of class loses it's value when the class being used as graph attribute
Date Mon, 30 Jun 2014 09:16:19 GMT
The code base is huge but sharing the snapshot of it which I think might give
you some idea . Here is my class Player which is supposed to be my vertex
attribute : 

*class Player(var RvalRdd: RDD[((Int, Int), Double)], Slope_m: Double)
extends Serializable {
//Some code here

As you can see this takes a constructor parameter one of which is an RDD.

Here is my main execution class where I create vertex with attributes of

object RLSparkExceution2 {
  // Some code here

  val sc = new SparkContext("local", "Multiplayer")

 * val p1 = new Player(initRvalRdd(sc), m1) //Instantiates my RDD field*

  val p2 = new Player(initRvalRdd(sc), m2)

  val dummy_player = new Player(sc.parallelize(List(((0, 0), 0.0))), 0)
  val dummy_player_msg = new Player(sc.parallelize(List(((0, 0), 0.0))), 0)
  val players: RDD[(VertexId, Player)] = sc.parallelize(Array((1L, p1), (2L,
p2), (0L, dummy_player)))
  val connections: RDD[Edge[String]] = sc.parallelize(Array(Edge(1L, 0L,
"dummyConnect"), Edge(2L, 0L, dummyConnect")))
  var graph = Graph(players, connections)

  def main(args: Array[String]): Unit = {

      *println(p1.RvalRdd.count)  //Here this gets printed* 
     //*This one throws a nullpointer exception*
      graph.vertices.foreach(vtx =>if(vtx._1 !=0L) println(s"test to see
rval in beginnig ${vtx._1} :: ${*vtx._2.RvalRdd.count*}"))



View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message