spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tathagata Das <>
Subject Re: custom receiver in java
Date Tue, 01 Apr 2014 20:37:18 GMT
Unfortunately, there isnt a good Java-friendly way to define custom
receivers. However, I am currently refactoring the receiver interface to
make it more Java friendly and I hope to get that in Spark 1.0 release.

In the meantime, I would encourage you define the custom receiver in Scala.
If you are not comfortable with Scala, we can write all the core
functionality in Java as functions, and create a skeleton Scala custom
receiver that calls into the Java functions (shown below). Unfortunately,
you have to use scala compiler to compile this.


      class CustomReceiver(parameters: CustomReceiverParameters)
         extends NetworkReceiver[*<type of records>*]
         protected lazy val blocksGenerator: BlockGenerator =
           new BlockGenerator(StorageLevel.MEMORY_ONLY_SER_2)
*         val javaReceiver = new MyJavaReceiver(parameters, blockGenerator)
 // create an instance of java receiver object*

         protected def onStart() = {
*           javaReceiver.start()   // start the java receiver*

         protected def onStop() {
*           javaReceiver.stop()*

On Tue, Apr 1, 2014 at 7:54 AM, eric perler <> wrote:

> i would like to write a custom receiver to receive data from a Tibco RV
> subject
> i found this scala example..
> but i cant seem to find a java example
> does anybody know of a good java example for creating a custom receiver
> thx

View raw message