spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suraj Satishkumar Sheth <>
Subject RE: Actors and sparkcontext actions
Date Tue, 04 Mar 2014 09:20:03 GMT
Hi Ognen,
See if this helps. I was working on this :

class MyClass[T](sc : SparkContext, flag1 : Boolean, rdd : RDD[T], hdfsPath : String) extends
Actor {

  def act(){
    if(flag1) this.process()
    else this.count
  private def process(){
    //do the processing
  private def count(){
   //do the counting


Thanks and Regards,
Suraj Sheth

-----Original Message-----
From: Ognen Duzlevski [] 
Sent: 27 February 2014 01:09
Subject: Actors and sparkcontext actions

Can someone point me to a simple, short code example of creating a basic Actor that gets a
context and runs an operation such as .textFile.count? 
I am trying to figure out how to create just a basic actor that gets a message like this:

case class Msg(filename:String, ctx: SparkContext)

and then something like this:

class HelloActor extends Actor {
     import context.dispatcher

     def receive = {
         case Msg(fn,ctx) => {
             // get the count here!
             // cts.textFile(fn).count
         case _ => println("huh?")

Where I would want to do something like:

val conf = new
val sc = new SparkContext(conf)
val system = ActorSystem("mySystem")

val helloActor1 = system.actorOf( Props[ HelloActor], name = "helloactor1")
helloActor1 ! new Msg("test.json",sc)


View raw message