beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Etienne Chauchot (JIRA)" <>
Subject [jira] [Created] (BEAM-5212) Update native spark metrics
Date Fri, 24 Aug 2018 12:15:00 GMT
Etienne Chauchot created BEAM-5212:

             Summary: Update native spark metrics
                 Key: BEAM-5212
             Project: Beam
          Issue Type: Improvement
          Components: runner-spark
            Reporter: Etienne Chauchot
            Assignee: Amit Sela

Native engine metrics are updated on {{finishBundle}} by DoFnRunners. Spark uses {{SimpleDoFnRunner}}
that is in runner-core and thus contains nothing spark native. It might be good to extend
{{SimpleDoFnRunner}} to add native metrics support just like it was done in Flink with {{DoFnRunnerWithMetrics}}

This message was sent by Atlassian JIRA

View raw message