spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <>
Subject Cross cutting internal changes to launch scripts
Date Mon, 09 Mar 2015 19:53:00 GMT
Hey All,

Marcelo Vanzin has been working on a patch for a few months that
performs cross cutting clean-up and fixes to the way that Spark's
launch scripts work (including PySpark, spark submit, the daemon
scripts, etc.). The changes won't modify any public API's in terms of
how those scripts are invoked.

Historically, such patches have been difficult to test due to the
number of interactions between components and interactions with
external environments. I'd like to welcome people to test and/or code
review this patch in their own environment. This patch is the in the
very late stages of review and will likely be merged soon into master
(eventually 1.4).

I'll ping this thread again once it is merged and we can establish a
JIRA to encapsulate any issues. Just wanted to give a heads up as this
is one of the larger internal changes we've made to this
infrastructure since Spark 1.0

- Patrick

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message