spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veera (Jira)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-29021) NoSuchElementException: key not found: hostPath.spark-local-dir-5.options.path
Date Thu, 05 Dec 2019 05:43:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-29021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988487#comment-16988487
] 

Veera commented on SPARK-29021:
-------------------------------

Hi 

 

Even im faing the issue while submiitng the spark

 

/opt/spark/bin/spark-submit --master XXXXXXXXXXXXXXXXXXXXXXXXXXX/ \
--deploy-mode cluster \
--name spark-wordcount \
--conf spark.kubernetes.namespace=spark 
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spar-sa \
--conf spark.executor.instances=3 \
--conf spark.kubernetes.container.image.pullPolicy=Always \
--conf spark.kubernetes.container.image=container image latest version \
--conf spark.kubernetes.pyspark.pythonVersion=3 \
--conf spark.kubernetes.authenticate.submission.caCertFile=/etc/kubernetes/pki/ca.crt \
--conf spark.kubernetes.driver.volumes.hostPath.sparkwordcount.mount.path=/root \
--conf spark.kubernetes.driver.volumes.hostPath.sparkwordcount.mount.readOnly=false \
--conf spark.kubernetes.driver.volumes.hostPath.sparkwordcount.options.claimName=spark-wordcount-claim
\
/root/pyscripts/wordcount.py

 

 

Exception in thread "main" java.util.NoSuchElementException: hostPath.sparkwordcount.options.path
 at org.apache.spark.deploy.k8s.KubernetesVolumeUtils$MapOps$$anonfun$getTry$1.apply(KubernetesVolumeUtils.scala:107)
 at org.apache.spark.deploy.k8s.KubernetesVolumeUtils$MapOps$$anonfun$getTry$1.apply(KubernetesVolumeUtils.scala:107)
 at scala.Option.fold(Option.scala:158)
 at org.apache.spark.deploy.k8s.KubernetesVolumeUtils$MapOps.getTry(KubernetesVolumeUtils.scala:107)

> NoSuchElementException: key not found: hostPath.spark-local-dir-5.options.path
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-29021
>                 URL: https://issues.apache.org/jira/browse/SPARK-29021
>             Project: Spark
>          Issue Type: Bug
>          Components: Kubernetes
>    Affects Versions: 2.4.4
>            Reporter: Kent Yao
>            Priority: Major
>
> Mount hostPath has an issue:
> {code:java}
> Exception in thread "main" java.util.NoSuchElementException: key not found: hostPath.spark-local-dir-5.options.pathException
in thread "main" java.util.NoSuchElementException: key not found: hostPath.spark-local-dir-5.options.path
at scala.collection.MapLike.default(MapLike.scala:235) at scala.collection.MapLike.default$(MapLike.scala:234)
at scala.collection.AbstractMap.default(Map.scala:63) at scala.collection.MapLike.apply(MapLike.scala:144)
at scala.collection.MapLike.apply$(MapLike.scala:143) at scala.collection.AbstractMap.apply(Map.scala:63)
at org.apache.spark.deploy.k8s.KubernetesVolumeUtils$.parseVolumeSpecificConf(KubernetesVolumeUtils.scala:70)
at org.apache.spark.deploy.k8s.KubernetesVolumeUtils$.$anonfun$parseVolumesWithPrefix$1(KubernetesVolumeUtils.scala:43)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237) at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:321)
at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:977) at scala.collection.TraversableLike.map(TraversableLike.scala:237)
at scala.collection.TraversableLike.map$(TraversableLike.scala:230) at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:51)
at scala.collection.SetLike.map(SetLike.scala:104) at scala.collection.SetLike.map$(SetLike.scala:104)
at scala.collection.AbstractSet.map(Set.scala:51) at org.apache.spark.deploy.k8s.KubernetesVolumeUtils$.parseVolumesWithPrefix(KubernetesVolumeUtils.scala:33)
at org.apache.spark.deploy.k8s.KubernetesConf$.createDriverConf(KubernetesConf.scala:179)
at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:214)
at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:198)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:179) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:202)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:89) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:999)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1008) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message