spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From onmstester onmstester <>
Subject Fwd: use spark cluster in java web service
Date Thu, 01 Nov 2018 08:12:51 GMT
Refer: 1. Create a singleton SparkContext
at initialization of your cluster, the spark-context or spark-sql would be accessible through
a static method anywhere in your application. I recommend using Fair scheduling on your context,
to share resources among all input requests     SparkSession spark = SparkSession.builder().appName("Simple
 2. From now on, with sc or spark-sql object, something like sparkSql.sql("select * from test").collectAsList()
would be run as a spark job and returns result to your application Sent using Zoho Mail ============
Forwarded message ============ From : 崔苗(数据与人工智能产品开发部) <>
To : "user"<> Date : Thu, 01 Nov 2018 10:52:15 +0330 Subject :
use spark cluster in java web service ============ Forwarded message ============ Hi, we
want to use spark in our java web service , compute data in spark cluster according to request,now
we have two probles: 1、 how to get sparkSession of remote spark cluster (spark on yarn
mode) , we want to keep one sparkSession to execute all data compution; 2、how to submit
to remote spark cluster in java code instead of spark-submit , as we want to execute spark
code in reponse server; Thanks for any replys 0049003208 签名由 网易邮箱大师
定制 --------------------------------------------------------------------- To unsubscribe
View raw message