spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shubham <>
Subject Error in
Date Thu, 09 Aug 2018 05:48:01 GMT
Following the code snippets on  this thread
<>  , I got a working version of
XGBoost on pyspark. But one issues I am still facing is the following
line 92, in __init__    self._java_obj =
self.uid)  File
"/Users/ultrauser/Downloads/spark/python/pyspark/ml/", line 61, in
_new_java_obj    java_obj = getattr(java_obj, name)  File
line 1598, in __getattr__    raise Py4JError("{0} does not exist in the
ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator does not exist in the
JVMException ignored in: Traceback (most recent call last):  File
"/Users/ultrauser/Downloads/spark/python/pyspark/ml/", line 105,
in __del__   
SparkContext._active_spark_context._gateway.detach(self._java_obj)  File
line 2000, in detach    java_object._detach()AttributeError: 'NoneType'
object has no attribute '_detach'
>From what I read on StackOverflow and elsewhere, this looks like an issue of
jar locations. I have two jar files that are needed for this code to work
But I am not sure how to proceed. This is what I have tried so far
place the xgboost jar files in 
set the environment variables 
import osos.environ['PYSPARK_SUBMIT_ARGS'] = '--jars
/Users/ultrauser/Downloads/xgboost4j-spark-0.72.jar pyspark-shell'
But the error still persists. Is there something I am missing here. 

Sent from:
View raw message