spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From allonsy <>
Subject spark.streaming.maxRatePerPartition parameter: what are the benefits?
Date Thu, 13 Aug 2015 14:50:29 GMT
Hello everyone,

in the new Kafka Direct API, what are the benefits of setting a value for

In my case, I have 2 seconds batches consuming ~15k tuples from a topic
split into 48 partitions (4 workers, 16 total cores).

Is there any particular value I should be setting the parameter to, in order
to achieve better performances? And what happens if I don't set the value at

I could not find any detailed explanation about this.

Thank you!

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message