spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Patel <arunp.bigd...@gmail.com>
Subject Re: Graphframe Error
Date Thu, 07 Jul 2016 11:13:31 GMT
I have tied this already.  It does not work.

What version of Python is needed for this package?

On Wed, Jul 6, 2016 at 12:45 AM, Felix Cheung <felixcheung_m@hotmail.com>
wrote:

> This could be the workaround:
>
> http://stackoverflow.com/a/36419857
>
>
>
>
> On Tue, Jul 5, 2016 at 5:37 AM -0700, "Arun Patel" <
> arunp.bigdata@gmail.com> wrote:
>
> Thanks Yanbo and Felix.
>
> I tried these commands on CDH Quickstart VM and also on "Spark 1.6
> pre-built for Hadoop" version.  I am still not able to get it working.  Not
> sure what I am missing.  Attaching the logs.
>
>
>
>
> On Mon, Jul 4, 2016 at 5:33 AM, Felix Cheung <felixcheung_m@hotmail.com>
> wrote:
>
>> It looks like either the extracted Python code is corrupted or there is a
>> mismatch Python version. Are you using Python 3?
>>
>>
>> stackoverflow.com/questions/514371/whats-the-bad-magic-number-error
>>
>>
>>
>>
>>
>> On Mon, Jul 4, 2016 at 1:37 AM -0700, "Yanbo Liang" <ybliang8@gmail.com>
>> wrote:
>>
>> Hi Arun,
>>
>> The command
>>
>> bin/pyspark --packages graphframes:graphframes:0.1.0-spark1.6
>>
>> will automatically load the required graphframes jar file from maven
>> repository, it was not affected by the location where the jar file was
>> placed. Your examples works well in my laptop.
>>
>> Or you can use try with
>>
>> bin/pyspark --py-files ***/graphframes.jar --jars ***/graphframes.jar
>>
>> to launch PySpark with graphframes enabled. You should set "--py-files"
>> and "--jars" options with the directory where you saved graphframes.jar.
>>
>> Thanks
>> Yanbo
>>
>>
>> 2016-07-03 15:48 GMT-07:00 Arun Patel <arunp.bigdata@gmail.com>:
>>
>>> I started my pyspark shell with command  (I am using spark 1.6).
>>>
>>> bin/pyspark --packages graphframes:graphframes:0.1.0-spark1.6
>>>
>>> I have copied
>>> http://dl.bintray.com/spark-packages/maven/graphframes/graphframes/0.1.0-spark1.6/graphframes-0.1.0-spark1.6.jar
>>> to the lib directory of Spark as well.
>>>
>>> I was getting below error
>>>
>>> >>> from graphframes import *
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>> zipimport.ZipImportError: can't find module 'graphframes'
>>> >>>
>>>
>>> So, as per suggestions from similar questions, I have extracted the
>>> graphframes python directory and copied to the local directory where I am
>>> running pyspark.
>>>
>>> >>> from graphframes import *
>>>
>>> But, not able to create the GraphFrame
>>>
>>> >>> g = GraphFrame(v, e)
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>> NameError: name 'GraphFrame' is not defined
>>>
>>> Also, I am getting below error.
>>> >>> from graphframes.examples import Graphs
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>> ImportError: Bad magic number in graphframes/examples.pyc
>>>
>>> Any help will be highly appreciated.
>>>
>>> - Arun
>>>
>>
>>
>

Mime
View raw message