kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Consumers re-reading data issue
Date Thu, 13 Dec 2012 06:02:34 GMT
Casey,

Not sure what's happening there. Is this reproducible?

Thanks,

Jun

On Wed, Dec 12, 2012 at 9:52 AM, Sybrandy, Casey <
Casey.Sybrandy@six3systems.com> wrote:

> Hello,
>
> We have a strange issue on one of our systems that we just can't seem to
> figure out.  We loaded in 5000 records yesterday and have not loaded any
> since then into our brokers.  Examining the logs for the brokers, we
> confirmed that no data has been sent to those topics since we did the load
> yesterday.  However, since the data load yesterday, we've had the same
> record show up in our consumer logs 5 times.
>
> Offsets have been checked (We log everything INFO and above by default)
> and it doesn't look like that the consumers are reading from an older
> offset.  The only thing that I see that may cause this issue is a broker
> that sometimes is accessed by the consumers, but not all of the time.  It's
> an older broker that we don't use anymore for our project, but is still
> there for another project.  Until this morning, it did have the topic we
> were reading from on the broker.  I did remove the topic by deleting the
> directories and restarting the broker, so hopefully the consumers don't try
> to use it again.
>
> Could that be why we're seeing the duplicate records?  Or is there
> something else we need to check for?
>
> Casey
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message