kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Burroughs <chris.burrou...@gmail.com>
Subject Re: Solution for blocking fsync in 0.8
Date Tue, 19 Jun 2012 01:21:32 GMT
Thanks Jay.  This is a very helpful investigation!

On 05/24/2012 01:40 PM, Jay Kreps wrote:
> Unfortunately *any* call to fsync will block appends even in a background
> thread so how can we give control over physical disk persistence without
> introducing high latency for the producer? The answer is that the linux
> pdflush daemon actually does a very similar thing to our flush parameters.
> pdflush is a daemon running on every linux machine that controls the
> writing of buffered/cached data back to disk. It allows you to control the
> percentage of memory filled with dirty pages by giving it either a
> percentage of memory, a time out for any dirty page to be written, or a
> fixed number of dirty bytes.

This would however by necessity by a global setting right?  (Assuming
there is no /proc trickery to change per-pid pdflush behaviour)

View raw message