spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Davidson <>
Subject trouble with 'pip pyspark' pyspark.sql.functions. ³unresolved import² for col() and lit()
Date Wed, 04 Apr 2018 22:28:04 GMT
I am having trouble setting up my python3 virtualenv.

I created a virtualenv Œspark-2.3.0¹ Installed pyspark using pip how ever I
am not able to import pyspark.sql.functions. I get ³unresolved import² when
I try to import col() and lit()

from pyspark.sql.functions import *

I found if I download spark from apache and set SPARK_ROOT I can get my
juypter notebook to work. This is a very error prone work around. I am
having simiilar problem with my eclipse pyDev virtualenv

Any suggestions would be greatly appreciated


# pip show in virtualenv

(spark-2.3.0) $ pip show pyspark

Name: pyspark

Version: 2.3.0

Summary: Apache Spark Python API


Author: Spark Developers




Requires: py4j

(spark-2.3.0) $ 

(spark-2.3.0) $ ls 


# Jupyter Notebook
Export SPARK_ROOT=~/workSpace/spark/spark-2.3.0-bin-hadoop2.7

Eclipse pyDev virtual ENV

View raw message