kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Wang <john.w...@gmail.com>
Subject Re: Replay
Date Fri, 23 Mar 2012 02:13:01 GMT
Thanks Neha!

-John

On Thu, Mar 22, 2012 at 7:05 PM, Neha Narkhede <neha.narkhede@gmail.com>wrote:

> John,
>
> You can achieve that via the autooffset.reset="largest" config option
> in the consumer.
>
> Thanks,
> Neha
>
> On Thu, Mar 22, 2012 at 6:15 PM, John Wang <john.wang@gmail.com> wrote:
> > Hi Neha:
> >
> >     Can you point me to the api to always get data from tail?
> >
> > Thanks
> >
> > -John
> >
> > On Thu, Mar 22, 2012 at 5:02 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
> >
> >> Elben,
> >>
> >> Or sensei can provide a restart from tail API, that will delete the
> offset
> >> state from ZK and restart its Kafka consumer. Currently that is the best
> >> way to have a Kafka consumer restart fetching latest data.
> >>
> >> Thanks,
> >> Neha
> >> On Mar 22, 2012 4:39 PM, "John Wang" <john.wang@gmail.com> wrote:
> >>
> >> > I think in the newer kafka release, there is a tool to update the
> version
> >> > for a client.
> >> >
> >> > Jun, can you elaborate on that?
> >> >
> >> > Thanks
> >> >
> >> > -John
> >> >
> >> > On Thu, Mar 22, 2012 at 4:33 PM, Elben Shira <elbenshira@gmail.com>
> >> wrote:
> >> >
> >> > > We're using use kafka+sensei at our company and we'd love to be
> able to
> >> > > start at the end of the queue (latest entries). The use case is when
> >> > sensei
> >> > > goes down or has to be restarted, we only care about indexing the
> >> latest
> >> > > items.
> >> > >
> >> > > A workaround I guess is to stop sensei, manually change the high
> >> > watermark
> >> > > in kafka, then restart sensei.
> >> > >
> >> > > Elben
> >> > >
> >> > >
> >> > > On Thu, Mar 15, 2012 at 11:31 AM, Jun Rao <junrao@gmail.com>
wrote:
> >> > >
> >> > > > Casey,
> >> > > >
> >> > > > You can't get offsets using zk-based consumer now. We are still
> >> trying
> >> > to
> >> > > > figure about how to support that in the future.
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Jun
> >> > > >
> >> > > > On Thu, Mar 15, 2012 at 7:57 AM, Sybrandy, Casey <
> >> > > > Casey.Sybrandy@six3systems.com> wrote:
> >> > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > Just out of curiosity, how would you get the offsets when
using
> a
> >> > > > zk-based
> >> > > > > consumer?  I don't see anything that seems to let you do
this
> >> easily.
> >> > > > >
> >> > > > > Casey
> >> > > > >
> >> > > > > ________________________________________
> >> > > > > From: Joel Koshy [jjkoshy.w@gmail.com]
> >> > > > > Sent: Monday, March 12, 2012 5:51 PM
> >> > > > > To: kafka-users@incubator.apache.org
> >> > > > > Subject: Re: Replay
> >> > > > >
> >> > > > > With simple consumer, you can start from any valid offset
that
> you
> >> > keep
> >> > > > > track off.
> >> > > > >
> >> > > > > With the high level (zk-based) consumer: you will need to
stop
> all
> >> > > > > consumers, and then update the offsets in zookeeper (see
the
> >> > > > > ImportZkOffsets tool in the tools package) with the offsets
you
> >> want
> >> > to
> >> > > > > replay from before restarting your consumer.
> >> > > > >
> >> > > > > Thanks,
> >> > > > >
> >> > > > > Joel
> >> > > > >
> >> > > > > On Mon, Mar 12, 2012 at 12:09 PM, Jain, Kokil <
> JainK@bit-sys.com>
> >> > > wrote:
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > Sorry, new to Kafka. Is it possible for a consumer
to replay
> >> > messages
> >> > > > > from
> >> > > > > > an "offset" that may have been committed before?
> >> > > > > >
> >> > > > > > Thanks for the help,
> >> > > > > > Kokil Jain
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message