toree-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lrese...@apache.org
Subject incubator-toree git commit: [TOREE-397] Avoid hardcoding the deploy mode property
Date Mon, 27 Mar 2017 00:09:32 GMT
Repository: incubator-toree
Updated Branches:
  refs/heads/master 98229edf0 -> 092b90f40


[TOREE-397] Avoid hardcoding the deploy mode property

By checking if a proper spark.submit.deployMode property
is available in the sparkConf enables choosing the deploy
mode by using system properties such as $SPARK_OPTS.

This change continues to use client mode if no configuration
was provided.

Closes #117


Project: http://git-wip-us.apache.org/repos/asf/incubator-toree/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-toree/commit/092b90f4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-toree/tree/092b90f4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-toree/diff/092b90f4

Branch: refs/heads/master
Commit: 092b90f404f39e8e464b6e29949d2e8670bc4c7f
Parents: 98229ed
Author: Luciano Resende <lresende@apache.org>
Authored: Wed Mar 22 11:26:26 2017 -0700
Committer: Luciano Resende <lresende@apache.org>
Committed: Sun Mar 26 17:07:21 2017 -0700

----------------------------------------------------------------------
 .../src/main/scala/org/apache/toree/kernel/api/Kernel.scala | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-toree/blob/092b90f4/kernel/src/main/scala/org/apache/toree/kernel/api/Kernel.scala
----------------------------------------------------------------------
diff --git a/kernel/src/main/scala/org/apache/toree/kernel/api/Kernel.scala b/kernel/src/main/scala/org/apache/toree/kernel/api/Kernel.scala
index 952842c..417983b 100644
--- a/kernel/src/main/scala/org/apache/toree/kernel/api/Kernel.scala
+++ b/kernel/src/main/scala/org/apache/toree/kernel/api/Kernel.scala
@@ -366,8 +366,13 @@ class Kernel (
   // TODO: Think of a better way to test without exposing this
   protected[toree] def createSparkConf(conf: SparkConf) = {
 
-    logger.info("Setting deployMode to client")
-    conf.set("spark.submit.deployMode", "client")
+    if(conf.contains("spark.submit.deployMode")) {
+      logger.info("Utilizing deploy mode: " + conf.get("spark.submit.deployMode"))
+    } else {
+      logger.info("Setting deployMode to client")
+      conf.set("spark.submit.deployMode", "client")
+    }
+
     conf
   }
 


Mime
View raw message