spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darren Govoni <>
Subject RE: AMQP extension for Apache Spark Streaming (messaging/IoT)
Date Sun, 03 Jul 2016 12:31:21 GMT

This is fantastic news.

Sent from my Verizon 4G LTE smartphone

-------- Original message --------
From: Paolo Patierno <> 
Date: 7/3/16  4:41 AM  (GMT-05:00) 
Subject: AMQP extension for Apache Spark Streaming (messaging/IoT) 

Hi all,

I'm working on an AMQP extension for Apache Spark Streaming, developing a reliable receiver
for that. 

 MQTT support (I see it in the Apache Bahir repository), another messaging/IoT protocol 
could be very useful for the Apache Spark Streaming ecosystem. Out there a lot of broker (with
"store and forward" mechanism) support AMQP as first citizen protocol other than the Apache
Qpid Dispatch Router that is based on that for message routing.
Currently the source code is in my own GitHub account and it's in a early stage; the first
step was just having something working end-to-end. I'm going to add feature like QoS and flow
control in AMQP terms very soon. I was inspired by the spark-packages directories structure
using Scala (as main language) and SBT (as build tool).

What do 
you think about that ?

Looking forward to hear from you.

View raw message