spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ClockSlave <>
Subject Error in
Date Thu, 09 Aug 2018 06:00:44 GMT
		Following the code snippets on  this thread
<>  , I got a working version of
XGBoost on pyspark. But one issue I am still facing is the following
line 92, in __init__    self._java_obj =
self.uid)  File
"/Users/ultrauser/Downloads/spark/python/pyspark/ml/", line 61, in
_new_java_obj    java_obj = getattr(java_obj, name)  File
line 1598, in __getattr__    raise Py4JError("{0} does not exist in the
ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator does not exist in the
JVMException ignored in: Traceback (most recent call last):  File
"/Users/ultrauser/Downloads/spark/python/pyspark/ml/", line 105,
in __del__   
SparkContext._active_spark_context._gateway.detach(self._java_obj)  File
line 2000, in detach    java_object._detach()AttributeError: 'NoneType'
object has no attribute '_detach'
>From what I read on StackOverflow and elsewhere, this looks like an issue of
jar locations. I have two jar files that are needed for this code to work
But I am not sure how to proceed. This is what I have tried so far
place the xgboost jar files in 
set the environment variables 
import osos.environ['PYSPARK_SUBMIT_ARGS'] = '--jars
/Users/ultrauser/Downloads/xgboost4j-spark-0.72.jar pyspark-shell'
Place the jar files in $SPARK_HOME/jars
But the error still persists. Is there something I am missing here?

Sent from:
View raw message