spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergio Fernández <wik...@apache.org>
Subject Re: ImportError: No module named numpy
Date Thu, 02 Jun 2016 09:31:36 GMT
On Thu, Jun 2, 2016 at 9:59 AM, Bhupendra Mishra <bhupendra.mishra@gmail.com
> wrote:
>
> and i have already exported environment variable in spark-env.sh as
> follows.. error still there  error: ImportError: No module named numpy
>
> export PYSPARK_PYTHON=/usr/bin/python
>

According the documentation at
http://spark.apache.org/docs/latest/configuration.html#environment-variables
the PYSPARK_PYTHON environment variable is for poniting to the Python
interpreter binary.

If you check the programming guide
https://spark.apache.org/docs/0.9.0/python-programming-guide.html#installing-and-configuring-pyspark
it says you need to add your custom path to PYTHONPATH (the script
automatically adds the bin/pyspark there).

So typically in Linux you would need to add the following (assuming you
installed numpy there):

export PYTHONPATH=$PYTHONPATH:/usr/lib/python2.7/dist-packages

Hope that helps.




> On Thu, Jun 2, 2016 at 12:04 AM, Julio Antonio Soto de Vicente <
> julio@esbet.es> wrote:
>
>> Try adding to spark-env.sh (renaming if you still have it with .template
>> at the end):
>>
>> PYSPARK_PYTHON=/path/to/your/bin/python
>>
>> Where your bin/python is your actual Python environment with Numpy
>> installed.
>>
>>
>> El 1 jun 2016, a las 20:16, Bhupendra Mishra <bhupendra.mishra@gmail.com>
>> escribió:
>>
>> I have numpy installed but where I should setup PYTHONPATH?
>>
>>
>> On Wed, Jun 1, 2016 at 11:39 PM, Sergio Fernández <wikier@apache.org>
>> wrote:
>>
>>> sudo pip install numpy
>>>
>>> On Wed, Jun 1, 2016 at 5:56 PM, Bhupendra Mishra <
>>> bhupendra.mishra@gmail.com> wrote:
>>>
>>>> Thanks .
>>>> How can this be resolved?
>>>>
>>>> On Wed, Jun 1, 2016 at 9:02 PM, Holden Karau <holden@pigscanfly.ca>
>>>> wrote:
>>>>
>>>>> Generally this means numpy isn't installed on the system or your
>>>>> PYTHONPATH has somehow gotten pointed somewhere odd,
>>>>>
>>>>> On Wed, Jun 1, 2016 at 8:31 AM, Bhupendra Mishra <
>>>>> bhupendra.mishra@gmail.com> wrote:
>>>>>
>>>>>> If any one please can help me with following error.
>>>>>>
>>>>>>  File
>>>>>> "/opt/mapr/spark/spark-1.6.1/python/lib/pyspark.zip/pyspark/mllib/__init__.py",
>>>>>> line 25, in <module>
>>>>>>
>>>>>> ImportError: No module named numpy
>>>>>>
>>>>>>
>>>>>> Thanks in advance!
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Cell : 425-233-8271
>>>>> Twitter: https://twitter.com/holdenkarau
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Sergio Fernández
>>> Partner Technology Manager
>>> Redlink GmbH
>>> m: +43 6602747925
>>> e: sergio.fernandez@redlink.co
>>> w: http://redlink.co
>>>
>>
>>
>


-- 
Sergio Fernández
Partner Technology Manager
Redlink GmbH
m: +43 6602747925
e: sergio.fernandez@redlink.co
w: http://redlink.co

Mime
View raw message