spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: spark 2.0 readStream from a REST API
Date Sun, 31 Jul 2016 15:19:45 GMT
On Sun, Jul 31, 2016 at 12:53 PM, Ayoub Benali
<benali.ayoub.info@gmail.com> wrote:

> I started playing with the Structured Streaming API in spark 2.0 and I am
> looking for a way to create streaming Dataset/Dataframe from a rest HTTP
> endpoint but I am bit stuck.

What a great idea! Why did I myself not think about this?!?!

> What would be the easiest way to hack around it ? Do I need to implement the
> Datasource API ?

Yes and perhaps Hadoop API too, but not sure which one exactly since I
haven't even thought about it (not even once).

> Are there examples on how to create a DataSource from a REST endpoint ?

Never heard of one.

I'm hosting a Spark/Scala meetup this week so I'll definitely propose
it as a topic. Thanks a lot!

Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message