spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dillon Bostwick <dillon.bostw...@databricks.com>
Subject Re: Back pressure not working on streaming
Date Wed, 02 Jan 2019 03:50:19 GMT
Unsubscribe

On Tue, Jan 1, 2019 at 10:03 PM JF Chen <darouwan@gmail.com> wrote:

> I have set  spark.streaming.backpressure.enabled to true,  spark.streaming.backpressure.initialRate
> to 10.
> Once my application started, it received 32 million messages on first
> batch.
> My application runs every 300 seconds, with 32 kafka partition. So what's
> is the max rate if I set initial rate to 10?
>
> Thanks!
>
>
> Regard,
> Junfeng Chen
>
-- 

Dillon Bostwick
Solutions Engineer
Databricks <http://databricks.com/>
678-770-5344

Mime
View raw message