kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Cai <h...@pinterest.com.INVALID>
Subject HDD or SSD or EBS for kafka brokers in Amazon EC2
Date Tue, 02 Jun 2015 16:37:49 GMT
We have been hosting kafka brokers in Amazon EC2 and we are using EBS
disk.  But periodically we were hit by long I/O wait time on EBS in some
Availability Zones.

We are thinking to change the instance types to a local HDD or local SSD.
HDD is cheaper and bigger and seems quite fit for the Kafka use case which
is mostly sequential read/write, but some early experiments show the HDD
cannot catch up with the message producing speed since there are many
topic/partitions on the broker which actually makes the disk I/O more
randomly accessed.

How are people's experience of choosing disk types on Amazon?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message