spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Nist <>
Subject Re: Query REST web service with Spark?
Date Wed, 01 Apr 2015 00:06:38 GMT
Here are a few ways to achieve what your loolking to do:

Spark Job Server - -

defines a REST API for Spark

Hue -

Spark Kernel project:

> The Spark Kernel's goal is to serve as the foundation for interactive
> applications. The project provides a client library in Scala that abstracts
> connecting to the kernel (containing a Spark Context), which can be
> embedded into a web application. We demonstrated this at StataConf when we
> embedded the Spark Kernel client into a Play application to provide an
> interactive web application that communicates to Spark via the Spark Kernel
> (hosting a SparkContext).

Hopefully one of those will give you what your looking for.


On Tue, Mar 31, 2015 at 5:06 PM, Burak Yavuz <> wrote:

> Hi,
> If I recall correctly, I've read people integrating REST calls to Spark
> Streaming jobs in the user list. I don't imagine any cases for why it
> shouldn't be possible.
> Best,
> Burak
> On Tue, Mar 31, 2015 at 1:46 PM, Minnow Noir <> wrote:
>> We have have some data on Hadoop that needs augmented with data only
>> available to us via a REST service.  We're using Spark to search for, and
>> correct, missing data. Even though there are a lot of records to scour for
>> missing data, the total number of calls to the service is expected to be
>> low, so it would be ideal to do the whole job in Spark as we scour the data.
>> I don't see anything obvious in the API or on Google relating to making
>> REST calls from a Spark job.  Is it possible?
>> Thanks,
>> Alec

View raw message