beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Etienne Chauchot (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-5212) Update native spark metrics
Date Fri, 24 Aug 2018 12:15:00 GMT
Etienne Chauchot created BEAM-5212:
--------------------------------------

             Summary: Update native spark metrics
                 Key: BEAM-5212
                 URL: https://issues.apache.org/jira/browse/BEAM-5212
             Project: Beam
          Issue Type: Improvement
          Components: runner-spark
            Reporter: Etienne Chauchot
            Assignee: Amit Sela


Native engine metrics are updated on {{finishBundle}} by DoFnRunners. Spark uses {{SimpleDoFnRunner}}
that is in runner-core and thus contains nothing spark native. It might be good to extend
{{SimpleDoFnRunner}} to add native metrics support just like it was done in Flink with {{DoFnRunnerWithMetrics}}
class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message