kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjko...@gmail.com>
Subject Re: get offsets?
Date Thu, 22 Sep 2011 00:14:41 GMT
Hi Taylor,

This is an FAQ, that was asked some time ago as well:
http://mail-archives.apache.org/mod_mbox/incubator-kafka-users/201108.mbox/%3CCAFbh0Q3QgsYXn4kjTTh0zZ9DJZ7J14tM5g24RFJSihFfbqDNyw@mail.gmail.com%3E

GetOffsetShell doesn't return the offset of every message. It returns the
offset of the first message in every segment file. If you provide a time,
the offsets you get back are based on the last modified time of the segment
files.

It may be a good idea to paraphrase the above to be displayed with the
tool's help message.

Thanks,

Joel

On Wed, Sep 21, 2011 at 4:18 PM, Taylor Gautier <tgautier@tagged.com> wrote:
> Hi,
>
> Using Kafka 0.6.
>
> I'm trying to use get offsets but it doesn't seem to work as I expect.  I
> have  a test topic that has some messages in it.  Here's the output of a
> test client that starts from offset 0 and prints all messages/offsets for
> the topic:
>
> Consumed message:foo offset: 12
> Consumed message:bar offset: 24
> Consumed message:foo offset: 36
> Consumed message:bar offset: 48
> Consumed message:hello offset: 62
> Consumed message:world offset: 76
>
> Here's a class to print the last n offsets:
>
> public class SimpleConsumerDemo
>
> {
>
>  public static void main(String[] args)
>
>  {
>
>    SimpleConsumer simpleConsumer = new SimpleConsumer("localhost", 9092,
> 1000, 1024);
>
>
>    long[] offsets = simpleConsumer.getOffsetsBefore("test", 0, -1L, 1);
>
>    for (long l : offsets) {
>
>        System.out.println("offset: " + l);
>
>    }
>
>  }
>
> }
>
>
> Running this as above, with 1 offset, yields expected results:
>
> ----output-----
>
> offset: 76
>
>
> However, asking for 3 offsets yields unexpected results:
>
>
> change to: long[] offsets = simpleConsumer.getOffsetsBefore("test", 0, -1L,
> 3);
>
> ----output-----
>
> offset: 76
>
> offset: 0
>
>
> I expected:
>
>
> ----output-----
>
> offset: 76
>
> offset: 62
>
> offset: 48
>
> Any idea why I did not get what I was looking for/what I am doing wrong?
>

Mime
View raw message