spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Davies Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-3842) Remove the hacks for Python callback server in py4j
Date Wed, 08 Oct 2014 00:10:34 GMT
Davies Liu created SPARK-3842:
---------------------------------

             Summary: Remove the hacks for Python callback server in py4j 
                 Key: SPARK-3842
                 URL: https://issues.apache.org/jira/browse/SPARK-3842
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, Streaming
            Reporter: Davies Liu
            Priority: Minor


There are three hacks  while create Python API for Streaming (https://github.com/apache/spark/pull/2538)
:

1. daemonize the callback server thread, by 'thread.daemon = True' before start it  https://github.com/bartdag/py4j/issues/147
2. let callback server bind to random port, then update the Java callback client with real
port. https://github.com/bartdag/py4j/issues/148
3. start the callback server later. https://github.com/bartdag/py4j/issues/149



These hacks should be removed after py4j has fix these issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message