spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lalit1303 <>
Subject Re: Manipulating RDDs within a DStream
Date Fri, 31 Oct 2014 06:48:19 GMT

Since, the cassandra object is not serializable you can't open the
connection on driver level and access the object inside foreachRDD (i.e. at
worker level).
You have to open connection inside foreachRDD only, perform the operation
and then close the connection.

For example:

 wordCounts.foreachRDD( rdd => {

       val arr = rdd.toArray
       OPEN cassandra connection
       store arr 
       CLOSE cassandra connection



Lalit Yadav
View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message