kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Homer Strong <homer.str...@gmail.com>
Subject MirrorMaker doesn't exit on exception?
Date Tue, 02 Jul 2013 22:51:32 GMT
Hello,

I've noticed that when a broker in the target cluster restarts (i.e.
immediately comes back online), the MirrorMaker runs into a FATAL exception
but the process hangs indefinitely instead of exiting. This seems to me
like undesirable behavior but I'm not sure that it's a bug. I'd like for
the MirrorMaker process to exit so that it can be restarted, since it is
run under supervision.

We're on Kafka 0.7.2, and this MirrorMaker uses the sync producer and is
configured with a broker list.

What I typically find in the logs is something like this:

[2013-07-02 20:56:35,431]  99283617 [mirrormaker-0] INFO
 kafka.producer.SyncProducer  - Disconnecting from localhost:9094
[2013-07-02 20:56:35,434]  99283620 [mirrormaker-0] FATAL
kafka.tools.MirrorMaker$MirrorMakerThread  - %s stream unexpectedly exited.
java.io.IOException: Broken pipe
...
[2013-07-02 20:56:35,469]  99283655 [mirrormaker-0] INFO
 kafka.tools.MirrorMaker$MirrorMakerThread  - Stopped thread mirrormaker-0.

and then the process hangs! Only when I kill it does the MirrorMaker
complete the shutdown process, e.g.:

[2013-07-02 22:34:53,478]  508405 [Thread-2] INFO
 kafka.consumer.ZookeeperConsumerConnector  -
MirrorMaker_zkkafka-0.uswest2n.yb0t.com-1372803985231-10bd3698
ZKConsumerConnector
 shutting down
[2013-07-02 22:34:53,479]  508406 [Thread-2] INFO
 kafka.consumer.ZookeeperTopicEventWatcher  - Shutting down topic event
watcher.
[2013-07-02 22:34:53,479]  508406 [ZkClient-EventThread-19-localhost:2181]
INFO  org.I0Itec.zkclient.ZkEventThread  - Terminate ZkClient event thread.
[2013-07-02 22:34:53,482]  508409 [main-EventThread] INFO
 org.apache.zookeeper.ClientCnxn  - EventThread shut down
[2013-07-02 22:34:53,482]  508409 [Thread-2] INFO
 org.apache.zookeeper.ZooKeeper  - Session: 0x13ea3d122570085 closed
[2013-07-02 22:34:53,484]  508411 [Thread-2] INFO
 kafka.utils.KafkaScheduler  - force shutdown scheduler
Kafka-consumer-autocommit-
[2013-07-02 22:34:53,485]  508412 [FetchRunnable-0] INFO
 kafka.consumer.FetcherRunnable  - FecherRunnable
Thread[FetchRunnable-0,5,main] interrupted
[2013-07-02 22:34:53,486]  508413 [FetchRunnable-0] INFO
 kafka.consumer.FetcherRunnable  - stopping fetcher FetchRunnable-0 to host
zkkafka-0.uswest2n.yb0t.com
[2013-07-02 22:34:53,490]  4254397417 [kafka-processor-0] INFO
 kafka.network.Processor  - Closing socket connection to /10.249.66.10.
[2013-07-02 22:34:53,499]  508426 [ZkClient-EventThread-12-localhost:2181]
INFO  org.I0Itec.zkclient.ZkEventThread  - Terminate ZkClient event thread.
[2013-07-02 22:34:53,503]  508430 [Thread-2] INFO
 org.apache.zookeeper.ZooKeeper  - Session: 0x13ea3d122570084 closed
[2013-07-02 22:34:53,503]  508430 [main-EventThread] INFO
 org.apache.zookeeper.ClientCnxn  - EventThread shut down
[2013-07-02 22:34:53,504]  508431 [Thread-2] INFO
 kafka.consumer.ZookeeperConsumerConnector  -
MirrorMaker_zkkafka-0.uswest2n.yb0t.com-1372803985231-10bd3698
ZKConsumerConnector
 shut down completed
[2013-07-02 22:34:53,505]  508432 [Thread-2] INFO
 kafka.producer.ProducerPool  - Closing all sync producers
[2013-07-02 22:34:53,505]  508432 [Thread-2] INFO
 kafka.producer.SyncProducer  - Disconnecting from localhost:9094


Suggestions appreciated. Thanks in advance!

Homer

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message