kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Duffy <...@ianduffy.ie>
Subject Kafka Streams and AWS IOPS Credits
Date Wed, 24 May 2017 09:40:21 GMT
Hi All,

In the past, we experienced lots of problems with running Kafka Stream
Applications on AWS.

We've seen issues with state locking, memory spiking to 100% and the
instance dying, very
slow startup on pulling down initial rocksdbs and so on....

Today we realised that the instances were experiencing such high throughput
that the IOPS
credits on our EBSs storing the rocks EBS were completely used up resulting
in very slow IO
and most of our issues.

Sharing this as others might have the same problems.

We were running on EBSs of 100GB which give 300 "provisioned" IOPS (3
iops per 1gb).
Most cost effective solution for us is to just increase the EBS size to get
more "provisioned"
IOPS. Alternative solutions are to use i3.* instances or IO1 EBS volumes
but they are expensive.

Hope this helps somebody else,
Ian.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message