kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avi Flax <avi.f...@parkassist.com>
Subject Streams RocksDB State Store Disk Usage
Date Wed, 29 Jun 2016 15:33:54 GMT
Hello all,

I’ve been working on a Streams app and so far it’s going quite well. I’ve had the app
up and running in a staging environment for a few weeks, processing real data.

Today I logged into the server to check in on some things, and I found the disk was full.

I managed (with ncdu) to find that most of the unexpected usage was in:

/tmp/kafka-streams/my_app/0_0/rocksdb/my_intermediate_aggregate_topic

and specifically in files named LOG.old.{timestamp}. The files were 2.6 GB and 118 MB. Also,
LOG was 1.1 GB.

I checked the docs and searched the list (and Google) for info on Kafka Streams and/or RocksDB
disk usage, cleanup, etc, but didn’t find much.

So:

* Is this normal, expected behavior?

* Any recommendations or suggestions for dealing with this?

Thanks!
Avi
Mime
View raw message