spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shixiong Zhu (JIRA)" <>
Subject [jira] [Created] (SPARK-28605) Performance regression in SS's foreach
Date Fri, 02 Aug 2019 18:04:00 GMT
Shixiong Zhu created SPARK-28605:

             Summary: Performance regression in SS's foreach
                 Key: SPARK-28605
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 2.4.3
            Reporter: Shixiong Zhu

When "" return "false", ForeachSink v1 will skip the whole partition without
reading data. But in ForeachSink v2, due to the API limitation, it needs to read the whole
partition even if all data just gets dropped.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message