spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Holway <andrew.hol...@otternetworks.de>
Subject Re: Python - Spark Cassandra Connector on DC/OS
Date Tue, 01 Nov 2016 09:32:33 GMT
Sorry: Spark 2.0.0

On Tue, Nov 1, 2016 at 10:04 AM, Andrew Holway <
andrew.holway@otternetworks.de> wrote:

> Hello,
>
> I've been getting pretty serious with DC/OS which I guess could be
> described as a somewhat polished distribution of Mesos. I'm not sure how
> relevant DC/OS is to this problem.
>
> I am using this pyspark program to test the cassandra connection:
> http://bit.ly/2eWAfxm (github)
>
> I can that the df.printSchema() method is working ok but the df.show()
> method is breaking with this error:
>
> Traceback (most recent call last):
>   File "/mnt/mesos/sandbox/squeeze.py", line 28, in <module>
>     df.show()
>   File "/opt/spark/dist/python/lib/pyspark.zip/pyspark/sql/dataframe.py",
> line 287, in show
>   File "/opt/spark/dist/python/lib/py4j-0.10.1-src.zip/py4j/java_gateway.py",
> line 933, in __call__
>   File "/opt/spark/dist/python/lib/pyspark.zip/pyspark/sql/utils.py",
> line 63, in deco
>   File "/opt/spark/dist/python/lib/py4j-0.10.1-src.zip/py4j/protocol.py",
> line 312, in get_return_value
> py4j.protocol.Py4JJavaError: An error occurred while calling
> o33.showString.
>
> Full output to stdout and stderr. : http://bit.ly/2f80f9e (gist)
>
> Versions:
>
> Spark 2.0.1
> Python Version: 3.4.3 (default, Sep 14 2016, 12:36:27)
> [cqlsh 5.0.1 | Cassandra 2.2.8 | CQL spec 3.3.1 | Native protocol v4]
> DC/OS v.1.8.4
>
> Cheers,
>
> Andrew
>
> --
> Otter Networks UG
> http://otternetworks.de
> Gotenstra├če 17
> 10829 Berlin
>



-- 
Otter Networks UG
http://otternetworks.de
Gotenstra├če 17
10829 Berlin

Mime
View raw message