spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Masiero Vanzin (Jira)" <>
Subject [jira] [Created] (SPARK-29396) Extend Spark plugin interface to driver
Date Tue, 08 Oct 2019 19:42:00 GMT
Marcelo Masiero Vanzin created SPARK-29396:

             Summary: Extend Spark plugin interface to driver
                 Key: SPARK-29396
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Marcelo Masiero Vanzin

Spark provides an extension API for people to implement executor plugins, added in SPARK-24918
and later extended in SPARK-28091.

That API does not offer any functionality for doing similar things on the driver side, though.
As a consequence of that, there is not a good way for the executor plugins to get information
or communicate in any way with the Spark driver.

I've been playing with such an improved API for developing some new functionality. I'll file
a few child bugs for the work to get the changes in.

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message