kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugen Dueck <eu...@tworks.co.jp>
Subject Re: log.dirs and SSDs
Date Thu, 12 Mar 2020 00:15:19 GMT
So there is not e.g. a single thread responsible per directory in log.dirs that could become
a bottleneck relative to SSD throughput of GB/s?

This is in fact the case for Apache Pulsar, and the openmessaging benchmark uses 4 directories
on the same SSD to increase throughput.

________________________________
差出人: Peter Bukowinski <pmbuko@gmail.com>
送信日時: 2020年3月12日 8:51
宛先: users@kafka.apache.org <users@kafka.apache.org>
件名: Re: log.dirs and SSDs

> On Mar 11, 2020, at 4:28 PM, Eugen Dueck <eugen@tworks.co.jp> wrote:
>
> So log.dirs should contain only one entry per HDD disk, to avoid random seeks.
> What about SSDs? Can throughput be increased by specifying multiple directories on the
same SSD?


Given a constant number of partitions, I don’t see any advantage to splitting partitions
among multiple log directories vs. keeping them all in one (per disk). You’d still have
the same total number of topic-partition directories and the same number of topic-partition
leaders.

If you want to increase throughput, focus on using the appropriate number of partitions.

—
Peter Bukowinski
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message