spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <hamishberri...@tutanota.com>
Subject Re: RESTful Operations
Date Mon, 20 Jan 2020 08:34:01 GMT
Sorry didn't explain well. Livy seems to me similar to job server by interacting through restful
api interface for  job submission. The task I want to achieve is more like general restful
api interface where user would provide some parameters for business operations (a bit like
readText or a custom date source with embedded http). So when the spark receive those requests
in e.g custom date source, the spark job can then perform normal operations such as  aggregation,
window functions, and so on. 

Thank you for the suggestions and help. 

    


Jan 20, 2020, 00:26 by chris.teoh@gmail.com:

> Maybe something like Livy, otherwise roll your own REST API and have it start a Spark
job.
>
> On Mon, 20 Jan 2020 at 06:55, <> hamishberridge@tutanota.com> > wrote:
>
>> I am new to Spark. The task I want to accomplish is let client send http requests,
then spark process that request for further operations. However  searching Spark's website
docs 
>>
>>     https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package
>>
>>     >> https://spark.apache.org/docs/latest/
>>
>> I do not find any places mentioning about this. Also most of the internet result
are more lated to spark job server. 
>>
>> Any places I should start if I want to use Spark for such purpose? 
>>
>> Thanks 
>>
>
>
> -- 
> Chris
>


Mime
View raw message