Why do't you explore Livy. You can use the Rest API to submit the jobs - https://community.hortonworks.com/articles/151164/how-to-submit-spark-application-through-livy-rest.html

On Thu, Nov 1, 2018 at 12:52 PM 崔苗(数据与人工智能产品开发部) <0049003208@znv.com> wrote:
we want to use spark in our java web service , compute data in spark cluster according to request,now we have two probles:
1、 how to get sparkSession of remote spark cluster (spark on yarn mode) , we want to keep one sparkSession to execute all data compution;
2、how to submit to remote spark cluster in java code instead of spark-submit , as we want to execute spark code in reponse server;

Thanks for any replys