spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maciej Szymkiewicz <>
Subject [ML][PYTHON] Collecting data in a class extending SparkSessionTestCase causes AttributeError:
Date Tue, 07 Mar 2017 03:29:53 GMT
Hi everyone,

It is a either to late or to early for me to think straight so please
forgive me if it is something trivial. I am trying to add a test case
extending SparkSessionTestCase to (example patch
attached). If test collects data, and there is another TestCase
extending extending SparkSessionTestCase executed before it, I get
AttributeError due to _jsc being None:


ERROR: test_foo (


Traceback (most recent call last):

  File "/home/spark/python/pyspark/ml/", line 1258, in test_foo

  File "/home/spark/python/pyspark/sql/", line 389, in collect

    with SCCallSiteSync(self._sc) as css:

  File "/home/spark/python/pyspark/", line 72, in __enter__


AttributeError: 'NoneType' object has no attribute 'setCallSite'


If TestCase is executed alone it seems to work just fine.

Can anyone reproduce this? Is there something obvious I miss here?


View raw message