spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Patierno <>
Subject AMQP extension for Apache Spark Streaming (messaging/IoT)
Date Sun, 03 Jul 2016 08:41:36 GMT
Hi all,

I'm working on an AMQP extension for Apache Spark Streaming, developing a reliable receiver
for that. 

 MQTT support (I see it in the Apache Bahir repository), another messaging/IoT protocol 
could be very useful for the Apache Spark Streaming ecosystem. Out there a lot of broker (with
"store and forward" mechanism) support AMQP as first citizen protocol other than the Apache
Qpid Dispatch Router that is based on that for message routing.
Currently the source code is in my own GitHub account and it's in a early stage; the first
step was just having something working end-to-end. I'm going to add feature like QoS and flow
control in AMQP terms very soon. I was inspired by the spark-packages directories structure
using Scala (as main language) and SBT (as build tool).

What do 
you think about that ?

Looking forward to hear from you.

View raw message