spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nsareen <>
Subject Re: Calling spark from a java web application.
Date Wed, 03 Dec 2014 00:48:48 GMT
We have a web application which talks to spark server.

This is how we have done the integration.
1) In the tomcat's classpath, add the spark distribution jar for spark code
to be available at runtime ( for you it would be Jetty).
2) In the Web application project, add the spark distribution jar in the
classpath ( Could be Java / Web project).
3) Setup the FAIR scheduling mode, which would help send parallel requests
from web application to the spark cluster.
4) In our application startup, initialize the connection to the spark
cluster. This is composed of creating the JavaSparkContext and making it
available throughout the web application, in case this needs to be the only
Driver Program required by the web application.
5) Using the JavaSpark Context, Create RDD's and make them available
globally to the web application code. 
6) invoke transformation / actions as required.

Hopefully this info is of some use..

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message