beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kyle Winkelman (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-5519) Spark Streaming Duplicated Encoding/Decoding Effort
Date Thu, 27 Sep 2018 20:38:00 GMT
Kyle Winkelman created BEAM-5519:
------------------------------------

             Summary: Spark Streaming Duplicated Encoding/Decoding Effort
                 Key: BEAM-5519
                 URL: https://issues.apache.org/jira/browse/BEAM-5519
             Project: Beam
          Issue Type: Bug
          Components: runner-spark
            Reporter: Kyle Winkelman
            Assignee: Kyle Winkelman


When using the SparkRunner in streaming mode. There is a call to groupByKey followed by a
call to updateStateByKey. BEAM-1815 fixed an issue where this used to cause 2 shuffles but
it still causes 2 encode/decode cycles.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message