kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Watzke <wat...@avast.com>
Subject clarification of the per-topic retention.bytes setting
Date Thu, 22 Jan 2015 12:38:40 GMT
Hi list,

please help me understand the per-topic retention.* setting (in kafka 
0.8.1.1) done by:

bin/kafka-topics.sh --zookeeper $ZK --alter --topic $TOPIC --config 
retention.bytes=VALUE

I understand from this:
http://search-hadoop.com/m/4TaT4E9f78/retention.bytes&subj=estimating+log+retention+bytes
that log.retention.bytes limit applies to a single partition, so 
basically each partition directory (in all directories listed in 
log.dirs) can take up at most "log.retention.bytes" bytes.

But what happens if I increase this limit for a single topic? Is this 
limit per-that-topic's-partition? So the overall topic size limit would be

LIMIT = retention.bytes * TOPIC'S_PARTITION_COUNT

and it could take up to LIMIT * TOPIC'S_REPLICATION_FACTOR of disk space?

Or is this setting "per-topic", meaning that retention.bytes property 
sets the upper overall topic size limit directly?

Thanks in advance!

-- 
David Watzke


Mime
View raw message