kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dana Powers <dana.pow...@rd.io>
Subject Re: offset commit api
Date Mon, 01 Sep 2014 00:20:32 GMT
following up on this -- I think the online API docs for OffsetCommitRequest
still incorrectly refer to client-side timestamps:

https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommitRequest

Wasn't that removed and now always handled server-side now?  Would one of
the devs mind updating the API spec wiki?

-Dana


On Mon, Aug 18, 2014 at 10:41 AM, Joel Koshy <jjkoshy.w@gmail.com> wrote:

> If you are using the high-level consumer then yes you would be under a
> group. If you are using the simple consumer then no. I just saw your
> earlier email in this thread - if you are using the simple consumer
> you can still use the offset commit API and set a group manually.
>
> On Mon, Aug 18, 2014 at 09:24:06AM -0700, Weide Zhang wrote:
> > Thanks Joel. Do you know if I'm using consumer group string
> >
> >
> > On Mon, Aug 4, 2014 at 3:16 PM, Joel Koshy <jjkoshy.w@gmail.com> wrote:
> > > Weide, 0.8.1.1 does not support offsets storage in Kafka. The brokers
> > > do support offset commit requests/fetches but simply forward to
> > > ZooKeeper - you can issue the offset commit and fetch requests to any
> > > broker. Kafka-backed consumer offsets is currently in trunk and will
> > > be released in 0.8.2.
> > >
> > > Thanks,
> > >
> > > Joel
> > >
> > > On Mon, Aug 04, 2014 at 02:57:02PM -0700, Weide Zhang wrote:
> > > > Hi
> > > >
> > > > It seems to me that 0.8.1.1 doesn't have the ConsumerMetadata API.
> So what
> > > > broker I should choose in order to commit and fetch offset
> information ?
> > > > Shall I use zookeeper for offset to manage it manually instead ?
> > > >
> > > > Thanks,
> > > >
> > > > Weide
> > > >
> > > >
> > > >
> > > > On Sun, Aug 3, 2014 at 4:34 PM, Weide Zhang <weoccc@gmail.com>
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm reading the offset management on the API link.
> > > > >
> > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit/FetchAPI
> > > > >
> > > > > I have a couple of questions regarding using the offset fetch and
> commit
> > > > > API in 0.8.1.1 ?
> > > > >
> > > > > 1. Is the new offset commit and fetch api usable in use in 0.8.1.1
> ?  Does
> > > > > 0.8.1.1 already support offset coordinator ?
> > > > >
> > > > > 2. what's the difference between old offsetrequest and new
> > > > > offsetfetchrequest ? It seems to me that the new api support per
> consumer
> > > > > group offset management fetch while old api doesn't. Also, what's
> the
> > > > > purpose of using a timestamp parameter in the fetch request ?
> > > > >
> > > > > 3. In 0.8.1.1, the OffsetCommitRequest uses
> OffsetMetadataAndError, could
> > > > > you tell me what's the purpose of the error parameter and metadata
> > > > > parameter in the request ?
> > > > >
> > > > > 4. Can I assume the offset management is purely independent of
> message
> > > > > consumption ? In other words, if I use a simple consumer to fetch
> message
> > > > > with random client id, can I still manually set some consumer
> group along
> > > > > with offset in the offset commit message ? Is that allowed ?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Weide
> > > > >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message