spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zenglong chen <czlong.kel...@gmail.com>
Subject Can pyspark use --archives to upload self-defined module than --py-files?
Date Thu, 25 Jul 2019 09:53:37 GMT
Hi,all,
     '--py-files' upload zip files and spark python will import module like
'./xxx.zip/module_a/a.py'.
So can i use '--archives a.zip#a' to upload zip file and spark worker
python will import like ''./a/module/xx“?
What is the difference between '--py-files' and '--archives a.zip#a' to
upload self-defined module?
Thanks for answer!

Mime
View raw message