spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-25233) Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure
Date Fri, 24 Aug 2018 16:52:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-25233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-25233:
------------------------------------

    Assignee: Apache Spark

> Give the user the option of specifying a fixed minimum message per partition per batch
when using kafka direct API with backpressure
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-25233
>                 URL: https://issues.apache.org/jira/browse/SPARK-25233
>             Project: Spark
>          Issue Type: Improvement
>          Components: DStreams
>    Affects Versions: 2.4.0
>            Reporter: Reza Safi
>            Assignee: Apache Spark
>            Priority: Major
>
> After SPARK-18371, it is guaranteed that there would be at least *one* message per
partition per batch using direct kafka API with backpressure when new messages exist in the
topics. It would be better if the user has the option of setting the minimum instead of just
a hard coded 1 limit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message