kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sybrandy, Casey" <Casey.Sybra...@Six3Systems.com>
Subject Consumers re-reading data issue
Date Wed, 12 Dec 2012 17:52:13 GMT
Hello,

We have a strange issue on one of our systems that we just can't seem to figure out.  We loaded
in 5000 records yesterday and have not loaded any since then into our brokers.  Examining
the logs for the brokers, we confirmed that no data has been sent to those topics since we
did the load yesterday.  However, since the data load yesterday, we've had the same record
show up in our consumer logs 5 times.

Offsets have been checked (We log everything INFO and above by default) and it doesn't look
like that the consumers are reading from an older offset.  The only thing that I see that
may cause this issue is a broker that sometimes is accessed by the consumers, but not all
of the time.  It's an older broker that we don't use anymore for our project, but is still
there for another project.  Until this morning, it did have the topic we were reading from
on the broker.  I did remove the topic by deleting the directories and restarting the broker,
so hopefully the consumers don't try to use it again.

Could that be why we're seeing the duplicate records?  Or is there something else we need
to check for?

Casey


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message