kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Inder Pall <inder.p...@gmail.com>
Subject Significance of multiple segment files in a partition
Date Tue, 25 Oct 2011 08:59:56 GMT
i am playing around with "log.file.size"(controls the size of a segment file
in a partition) and "log.retention.hours" with the following config.
log.file.size=500
log.retention.hours=168

Observation - i see multiple files getting generated within the same
partition.
Example : my topic name is revenue feed and i see the following

ls -lh /tmp/kafka-logs/revenuefeed-0/*
-rw-r--r-- 1 inder users 537 Oct 25 01:38
/tmp/kafka-logs/revenuefeed-0/00000000000000000000.kafka
-rw-r--r-- 1 inder users 512 Oct 25 01:39
/tmp/kafka-logs/revenuefeed-0/00000000000000000537.kafka

Questions
--------------
1. Shouldn't these two properties go hand in hand
2. Why would you want to have multiple files within a partition. Broker has
to store more info to figure the right file among a partition.
3. Is it to achieve mmap kinda optimization and allowing the broker to do
less I/O in case a feed is really huge or any thing else.

-- Inder

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message