spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Does Apache Spark use any Dependency Injection framework?
Date Sun, 02 Apr 2017 13:28:17 GMT
Hi All,

I am wondering if can get SparkConf
<https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/SparkConf.html>
object
through Dependency Injection? I currently use HOCON
<https://github.com/typesafehub/config/blob/master/HOCON.md> library to
store all key/value pairs required to construct SparkConf. The problem is
as I created multiple client jars(By client jars I mean the one we supply
for spark-submit to run our App) where each of them requiring its own
config It would be nice to have SparkConf created by the DI framework
depending on the client jar we want to run. I am assuming someone must have
done this?

Thanks!

Mime
View raw message