spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <>
Subject Re: Using Spark as web app backend
Date Tue, 24 Jun 2014 07:25:14 GMT

You could use sock.js / websockets on the front end, so you can notify the
user when the job is finished. You can regularly poll the URL of the job to
check its status from.your node.js app  - at the moment I do not know an
out of the box solution.

Nicer would be if your job sends a message via rabbitmq, kafka etc. once it
finishes and your node.js app receives this message and notifies via
websocket/ sock.js  the front end (i.e. the Javascript running in the HTML
page of your users browser). Perhaps you can also then add stomp.js - so
you can make sure the user receives the message in the browser. If I will
find time I will write an example app, because I think it is a relevant use

Best regards,

Le 24 juin 2014 09:13, "Jaonary Rabarisoa" <> a écrit :

> Hi all,
> So far, I run my spark jobs with spark-shell or spark-submit command. I'd
> like to go further and I wonder how to use spark as a backend of a web
> application. Specificaly, I want a frontend application ( build with nodejs
> )  to communicate with spark on the backend, so that every query from the
> frontend is rooted to spark. And the result from Spark are sent back to the
> frontend.
> Does some of you already experiment this kind of architecture ?
> Cheers,
> Jaonary

View raw message