kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Ormsbee <d...@datadoghq.com>
Subject Re: Timestamps
Date Wed, 07 Dec 2011 18:04:10 GMT
Hi Evan,

log.file.size in your kafka broker config file.

I don't think there's any way to trigger it from the consumer.

Dave

On Wed, Dec 7, 2011 at 12:59 PM, Evan Chan <ev@ooyala.com> wrote:

> Is there a setting that controls the log segment size?
>
> Better yet, is there a way to tell the broker (maybe from the consumer),
> look I want you to start a new log segment?
>
> thx,
> Evan
>
>
> On Wed, Dec 7, 2011 at 8:09 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > Mark,
> >
> > The SimpleConsumer class has an API for this - getOffsetsBefore(topic,
> > partition, time, maxNumOffsets).
> > Using this API, you can get the offset "closest" to a particular
> timestamp
> > at the log segment granularity. By
> > log segment granularity, I mean that it doesn't poke inside a log segment
> > and find the message right before or
> > right after the timestamp you want.
> >
> > Thus, it is just an approximation. The larger your log segment, the less
> > accuracy you will get using this API.
> >
> > Thanks,
> > Neha
> >
> > On Wed, Dec 7, 2011 at 7:54 AM, Mark <static.void.dev@gmail.com> wrote:
> >
> > > Is there any way to consume all messages starting at and/or between two
> > > timestamps?
> > >
> >
>
>
>
> --
> --
> *Evan Chan*
> Senior Software Engineer |
> ev@ooyala.com | (650) 996-4600
> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> @ooyala<http://www.twitter.com/ooyala>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message