spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Holway <andrew.hol...@otternetworks.de>
Subject Python - Spark Cassandra Connector on DC/OS
Date Tue, 01 Nov 2016 09:04:05 GMT
Hello,

I've been getting pretty serious with DC/OS which I guess could be
described as a somewhat polished distribution of Mesos. I'm not sure how
relevant DC/OS is to this problem.

I am using this pyspark program to test the cassandra connection:
http://bit.ly/2eWAfxm (github)

I can that the df.printSchema() method is working ok but the df.show()
method is breaking with this error:

Traceback (most recent call last):
  File "/mnt/mesos/sandbox/squeeze.py", line 28, in <module>
    df.show()
  File "/opt/spark/dist/python/lib/pyspark.zip/pyspark/sql/dataframe.py",
line 287, in show
  File
"/opt/spark/dist/python/lib/py4j-0.10.1-src.zip/py4j/java_gateway.py", line
933, in __call__
  File "/opt/spark/dist/python/lib/pyspark.zip/pyspark/sql/utils.py", line
63, in deco
  File "/opt/spark/dist/python/lib/py4j-0.10.1-src.zip/py4j/protocol.py",
line 312, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o33.showString.

Full output to stdout and stderr. : http://bit.ly/2f80f9e (gist)

Versions:

Spark 2.0.1
Python Version: 3.4.3 (default, Sep 14 2016, 12:36:27)
[cqlsh 5.0.1 | Cassandra 2.2.8 | CQL spec 3.3.1 | Native protocol v4]
DC/OS v.1.8.4

Cheers,

Andrew

-- 
Otter Networks UG
http://otternetworks.de
Gotenstra├če 17
10829 Berlin

Mime
View raw message