spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <>
Subject Re: spark 2.0 readStream from a REST API
Date Sun, 31 Jul 2016 15:19:45 GMT
On Sun, Jul 31, 2016 at 12:53 PM, Ayoub Benali
<> wrote:

> I started playing with the Structured Streaming API in spark 2.0 and I am
> looking for a way to create streaming Dataset/Dataframe from a rest HTTP
> endpoint but I am bit stuck.

What a great idea! Why did I myself not think about this?!?!

> What would be the easiest way to hack around it ? Do I need to implement the
> Datasource API ?

Yes and perhaps Hadoop API too, but not sure which one exactly since I
haven't even thought about it (not even once).

> Are there examples on how to create a DataSource from a REST endpoint ?

Never heard of one.

I'm hosting a Spark/Scala meetup this week so I'll definitely propose
it as a topic. Thanks a lot!

Jacek Laskowski
Mastering Apache Spark 2.0
Follow me at

To unsubscribe e-mail:

View raw message