kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Hørup Eskildsen <si...@sirupsen.com>
Subject Re: Flush configuration per topic
Date Wed, 16 Oct 2013 11:16:45 GMT
That's only the time delay. There seems to be no option to persist on every
message.


On Wed, Oct 16, 2013 at 12:05 AM, Jun Rao <junrao@gmail.com> wrote:

> In 0.8, we do have "log.flush.interval.ms.per.topic" (see
> http://kafka.apache.org/documentation.html#brokerconfigs for details).
>
> Thanks,
>
> Jun
>
>
> On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen
> <simon@sirupsen.com>wrote:
>
> > Do you mean that it's possible to override log configurations per topic
> in
> > trunk?
> >
> > Yeah, you're right. :-) I wasn't sure what to call it if not consistency,
> > even though I know that sort of has another meaning in this context.
> >
> >
> > On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
> >
> > > Yeah, looks like you are right, we don't have the per-topic override in
> > 0.8
> > > :-(
> > >
> > > All log configurations are overridable in trunk which will be 0.8.1.
> > >
> > > Just to be totally clear this setting does not impact consistency (i.e.
> > all
> > > replicas will have the same messages in the same order), nor even
> > > durability (if you have replication > 1), but just recoverability on a
> > > single server in the event of a hard machine crash.
> > >
> > > -Jay
> > >
> > >
> > > On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen
> > > <simon@sirupsen.com>wrote:
> > >
> > > > 0.8, we're not on master, but we definitely can be.
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 5:03 PM, Jay Kreps <jay.kreps@gmail.com>
> > wrote:
> > > >
> > > > > Hey Simon,
> > > > >
> > > > > What version of Kafka are you using?
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Tue, Oct 15, 2013 at 9:56 AM, Simon Hørup Eskildsen
> > > > > <simon@sirupsen.com>wrote:
> > > > >
> > > > > > Hi Kafkas!
> > > > > >
> > > > > > Reading through the documentation and code of Kafka, it seems
> there
> > > is
> > > > no
> > > > > > feature to set flushing interval (messages/time) for a specific
> > > topic.
> > > > I
> > > > > am
> > > > > > interested in this to get consistency for certain topics by
> > flushing
> > > > > after
> > > > > > every message, while having eventual consistency for other topics
> > > > > > (default).
> > > > > >
> > > > > > Would there be interest in such feature? Then I might be curious
> to
> > > > give
> > > > > > the Log module a dive and see if I can foster enough Scala to
add
> > > this.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > --
> > > > > > Simon
> > > > > > http://sirupsen.com/
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Simon
> > > > http://sirupsen.com/
> > > >
> > >
> >
> >
> >
> > --
> > Simon
> > http://sirupsen.com/
> >
>



-- 
Simon
http://sirupsen.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message