kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elben Shira <elbensh...@gmail.com>
Subject Re: Replay
Date Thu, 22 Mar 2012 23:33:11 GMT
We're using use kafka+sensei at our company and we'd love to be able to
start at the end of the queue (latest entries). The use case is when sensei
goes down or has to be restarted, we only care about indexing the latest
items.

A workaround I guess is to stop sensei, manually change the high watermark
in kafka, then restart sensei.

Elben


On Thu, Mar 15, 2012 at 11:31 AM, Jun Rao <junrao@gmail.com> wrote:

> Casey,
>
> You can't get offsets using zk-based consumer now. We are still trying to
> figure about how to support that in the future.
>
> Thanks,
>
> Jun
>
> On Thu, Mar 15, 2012 at 7:57 AM, Sybrandy, Casey <
> Casey.Sybrandy@six3systems.com> wrote:
>
> > Hello,
> >
> > Just out of curiosity, how would you get the offsets when using a
> zk-based
> > consumer?  I don't see anything that seems to let you do this easily.
> >
> > Casey
> >
> > ________________________________________
> > From: Joel Koshy [jjkoshy.w@gmail.com]
> > Sent: Monday, March 12, 2012 5:51 PM
> > To: kafka-users@incubator.apache.org
> > Subject: Re: Replay
> >
> > With simple consumer, you can start from any valid offset that you keep
> > track off.
> >
> > With the high level (zk-based) consumer: you will need to stop all
> > consumers, and then update the offsets in zookeeper (see the
> > ImportZkOffsets tool in the tools package) with the offsets you want to
> > replay from before restarting your consumer.
> >
> > Thanks,
> >
> > Joel
> >
> > On Mon, Mar 12, 2012 at 12:09 PM, Jain, Kokil <JainK@bit-sys.com> wrote:
> >
> > > Hi,
> > >
> > > Sorry, new to Kafka. Is it possible for a consumer to replay messages
> > from
> > > an "offset" that may have been committed before?
> > >
> > > Thanks for the help,
> > > Kokil Jain
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message