kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Maximum Topic Length in Kafka
Date Sat, 28 Nov 2015 21:53:31 GMT
Kafka server has a data retention policy based on either time or #.message
(e.g. Kafka brokers will automatically delete the oldest data segment if
its oldest data has been xx milliseconds ago, of if its total log size has
exceed yy MBs, with threshold values configurable).

The producer clients will not get affected by any of the log retention
policies on brokers, it will always be able to produce as long as the
server is alive. It is the server's responsibility to truncate data
according to the retention policy.

Guozhang

On Sat, Nov 28, 2015 at 10:34 AM, Marko Bonaći <marko.bonaci@sematext.com>
wrote:

> AFAIK there is no such notion as maximum length of a topic, i.e. offset has
> no limit, except Long.MAX_VALUE I think, which should be enough for a
> couple of lifetimes (9 * 10E18, or quintillion or million trillions).
>
> What would be the purpose of that, besides being a nice foot-gun :)
>
> Marko Bonaći
> Monitoring | Alerting | Anomaly Detection | Centralized Log Management
> Solr & Elasticsearch Support
> Sematext <http://sematext.com/> | Contact
> <http://sematext.com/about/contact.html>
>
> On Sat, Nov 28, 2015 at 2:13 PM, Debraj Manna <subharaj.manna@gmail.com>
> wrote:
>
> > Hi,
> >
> > Can some one please let me know the following:-
> >
> >
> >    1. Is it possible to specify maximum length of a particular topic ( in
> >    terms of number of messages ) in kafka ?
> >    2. Also how does Kafka behave when a particular topic gets full?
> >    3. Can the producer be blocked if a topic get full rather than
> deleting
> >    old messages?
> >
> > I have gone through the documentation
> > <http://kafka.apache.org/081/documentation.html#basic_ops_add_topic> but
> > could not find anything of what I am looking for.
> >
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message