spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Patierno <ppatie...@live.com>
Subject Python : StreamingContext isn't created properly
Date Wed, 03 Aug 2016 14:16:22 GMT
Hi all,

I wrote following function in Python :

def createStreamingContext():
    conf = (SparkConf().setMaster("local[2]").setAppName("my_app"))
    conf.set("spark.streaming.receiver.writeAheadLog.enable", "true")

    sc = SparkContext(conf=conf)
    ssc = StreamingContext(sc, 1)
    ssc.checkpoint("/tmp/spark-streaming")

    ....
    ....

    ssc
But I don't know why launching it with spark_submit gives me following error :

AttributeError: 'NoneType' object has no attribute 'checkpoint'

I can imagine that ssc wasn't created properly but ... why ?

Thanks,
Paolo.


Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience 		 	   		  
Mime
View raw message