spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From davies <davies....@gmail.com>
Subject Re: PySpark Driver from Jython
Date Fri, 11 Jul 2014 06:17:48 GMT
The function run in worker is serialized in driver, so the driver and worker
should be run in the same Python interpreter.

If you do not need c extension support, then Jython will be better than
CPython, because of the cost of serialization is much lower.

Davies



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/PySpark-Driver-from-Jython-tp7142p7269.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Mime
View raw message