kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bongyeon Kim <bongyeon....@gmail.com>
Subject Re: Conflict stored data in Zookeeper
Date Tue, 05 Aug 2014 05:43:44 GMT
Thanks for the reply.

Before I seen that log, I produced a lot of events for performance test.
(approximately 3G/min), and I have seen that log in an hour or two. and
I've got ERROR meesage frequently like below.

[2014-08-04 11:27:54,547] ERROR [ReplicaFetcherThread-0-6], Error in fetch
Name: FetchRequest; Version: 0; CorrelationId: 6263874; ClientId:
ReplicaFetcherThread-0-6; ReplicaId: 2; MaxWait: 500 ms; MinBytes: 1 bytes;
RequestInfo: [topicTRACE,13] ->
PartitionFetchInfo(10091892,1073741824),[topicTRACE,1] ->
PartitionFetchInfo(10174056,1073741824),[topicTRACE,9] ->
PartitionFetchInfo(10087558,1073741824),[topicTRACE,5] ->
PartitionFetchInfo(10148805,1073741824) (kafka.server.ReplicaFetcherThread)
java.io.EOFException: Received -1 when reading from channel, socket has
likely been closed.

While I was doing performance test, replications was getting shrinked, and
when it's done, I got my replications back. then I did preferred replica
election.

That's all I've done. I never start/stop brokers since I started it first
time.

Anyway, I'll try to restart my brokers like you said.


Thanks.




On Tue, Aug 5, 2014 at 1:26 PM, Joe Stein <joe.stein@stealth.ly> wrote:

> I have seen an issue similar to this but with the /controller node.
>
> I am going to update https://issues.apache.org/jira/browse/KAFKA-1387 with
> the steps to reproduce the issue I ran into right now.
>
> I don't know what steps caused what you ran into it is very odd that
> shouldn't happen.
>
> Were you doing anything with the cluster before this happened?
> starting/stopping nodes? any steps that might help reproduce it? any more
> info in any logs?
>
> I would recommend shutting down that node, make sure the znode is gone and
> starting it back up again.
>
> I also agree INFO is not a good status in that function, it is in a
> while(true) loop that may never end should at least be a WARN.
>
> /*******************************************
>  Joe Stein
>  Founder, Principal Consultant
>  Big Data Open Source Security LLC
>  http://www.stealth.ly
>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Mon, Aug 4, 2014 at 10:23 PM, Bongyeon Kim <bongyeon.kim@gmail.com>
> wrote:
>
> > Hi, everyone.
> >
> > I'm using 0.8.1.1, and I have 8 brokers and 3 topics each have 16
> > partitions and 3 replicas.
> >
> > I got unseen logs like below. this is occur every 5 seconds.
> >
> >
> > [2014-08-05 11:11:32,478] INFO conflict in /brokers/ids/2 data:
> >
> >
> {"jmx_port":9992,"timestamp":"1407204339990","host":"172.25.63.9","version":1,"port":9092}
> > stored data:
> >
> >
> {"jmx_port":9992,"timestamp":"1407204133312","host":"172.25.63.9","version":1,"port":9092}
> > (kafka.utils.ZkUtils$)
> > [2014-08-05 11:11:32,479] INFO I wrote this conflicted ephemeral node
> >
> >
> [{"jmx_port":9992,"timestamp":"1407204339990","host":"172.25.63.9","version":1,"port":9092}]
> > at /brokers/ids/2 a while back in a different session, hence I will
> backoff
> > for this node to be deleted by Zookeeper and retry (kafka.utils.ZkUtils$)
> >
> >
> > I hope to know the what makes this messages.
> > Is it OK that's not ERROR? How can I remove that message?
> >
> >
> > Thanks in adavnce.
> >
> > --
> > *Sincerely*
> > *,**Bongyeon Kim*
> >
> > Java Developer & Engineer
> > Seoul, Korea
> > Mobile:  +82-10-9369-1314
> > Email:  bongyeonkim@gmail.com
> > Twitter:  http://twitter.com/tigerby
> > Facebook:  http://facebook.com/tigerby
> > Wiki: http://tigerby.com
> >
>



-- 
*Sincerely*
*,**Bongyeon Kim*

Java Developer & Engineer
Seoul, Korea
Mobile:  +82-10-9369-1314
Email:  bongyeonkim@gmail.com
Twitter:  http://twitter.com/tigerby
Facebook:  http://facebook.com/tigerby
Wiki: http://tigerby.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message