spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emre Sevinc <emre.sev...@gmail.com>
Subject Re: Query REST web service with Spark?
Date Wed, 01 Apr 2015 07:50:12 GMT
Hello Minnow,

It is possible. You can for example use Jersey REST client to query a web
service and get its results in a Spark job. In fact, that's what we did
actually in a recent project (in a Spark Streaming application).

Kind regards,

Emre Sevinç
http://www.bigindustries.be/



On Tue, Mar 31, 2015 at 10:46 PM, Minnow Noir <minnownoir@gmail.com> wrote:

> We have have some data on Hadoop that needs augmented with data only
> available to us via a REST service.  We're using Spark to search for, and
> correct, missing data. Even though there are a lot of records to scour for
> missing data, the total number of calls to the service is expected to be
> low, so it would be ideal to do the whole job in Spark as we scour the data.
>
> I don't see anything obvious in the API or on Google relating to making
> REST calls from a Spark job.  Is it possible?
>
> Thanks,
>
> Alec
>



-- 
Emre Sevinc

Mime
View raw message