kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Surendranauth Hiraman <suren.hira...@velos.io>
Subject Re: question about new consumer offset management in 0.8.2
Date Thu, 05 Feb 2015 17:19:37 GMT
This is what I've found so far.

https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka

The high-level consumer just worked for me by setting offsets.storage =
kafka.

Scroll down to the offsets.* config params.

http://kafka.apache.org/documentation.html#consumerconfigs

-Suren



On Thu, Feb 5, 2015 at 12:16 PM, Jon Bringhurst <
jbringhurst@linkedin.com.invalid> wrote:

> There should probably be a wiki page started for this so we have the
> details in one place. The same question was asked on Freenode IRC a few
> minutes ago. :)
>
> A summary of the migration procedure is:
>
> 1) Upgrade your brokers and set dual.commit.enabled=false and
> offsets.storage=zookeeper (Commit offsets to Zookeeper Only).
> 2) Set dual.commit.enabled=true and offsets.storage=kafka and restart
> (Commit offsets to Zookeeper and Kafka).
> 3) Set dual.commit.enabled=false and offsets.storage=kafka and restart
> (Commit offsets to Kafka only).
>
> -Jon
>
> On Feb 5, 2015, at 9:03 AM, Jason Rosenberg <jbr@squareup.com> wrote:
>
> > Hi,
> >
> > For 0.8.2, one of the features listed is:
> >  - Kafka-based offset storage.
> >
> > Is there documentation on this (I've heard discussion of it of course)?
> >
> > Also, is it something that will be used by existing consumers when they
> > migrate up to 0.8.2?  What is the migration process?
> >
> > Thanks,
> >
> > Jason
>
>


-- 

SUREN HIRAMAN, VP TECHNOLOGY
Velos
Accelerating Machine Learning

54 West 40th Street, 3RD FLOOR
NEW YORK, NY 10018
T: @suren_h
E: suren.hiraman@v <suren.hiraman@sociocast.com>elos.io
W: www.velos.io

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message