spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JF Chen <darou...@gmail.com>
Subject Back pressure not working on streaming
Date Wed, 02 Jan 2019 03:02:42 GMT
I have set  spark.streaming.backpressure.enabled to true,
spark.streaming.backpressure.initialRate
to 10.
Once my application started, it received 32 million messages on first
batch.
My application runs every 300 seconds, with 32 kafka partition. So what's
is the max rate if I set initial rate to 10?

Thanks!


Regard,
Junfeng Chen

Mime
View raw message