kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aman Bazayev <abaza...@talenttech.com>
Subject Re: MirrorMaker consumer does not use broker.list property
Date Wed, 29 May 2013 06:24:48 GMT
Thanks for the response. What if I'm trying to set a broker.list for
consumer statically, is there a way to do that?

On Tue, May 28, 2013 at 9:05 PM, Jun Rao <junrao@gmail.com> wrote:

> Consumer only needs zk.connect, not broker.list.
>
> Thanks,
>
> Jun
>
>
> On Tue, May 28, 2013 at 11:31 AM, Aman Bazayev <abazayev@talenttech.com
> >wrote:
>
> > Hi,
> >
> > I'd like to bump this issue:
> >
> >
> https://mail-archives.apache.org/mod_mbox/kafka-users/201212.mbox/%3CFA0E8A0482D176408729D604142248F319D22CA7%40EXCHANGE14.actuate.com%3E
> > as I'm encountering the same problem.
> >
> > It seems that the MirrorMaker does the following:
> > 1) if zk.connect is defined for the consumer, MirrorMaker script ignores
> > broker.list value
> > 2) if zk.connect is not definied for the consumer, MirrorMaker fails with
> > an error message:
> >
> > [2013-05-28 11:24:57,457] INFO group1_ip-<hostname> Connecting to
> zookeeper
> > instance at null (kafka.consumer.ZookeeperConsumerConnector)
> > [2013-05-28 11:24:57,457] INFO Initiating client connection,
> > connectString=null sessionTimeout=6000
> > watcher=org.I0Itec.zkclient.ZkClient@c820344
> > (org.apache.zookeeper.ZooKeeper)
> > [2013-05-28 11:24:57,461] INFO Starting ZkClient event thread.
> > (org.I0Itec.zkclient.ZkEventThread)
> > Exception in thread "main" java.lang.NullPointerException
> >     at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
> >     at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
> >     at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
> >     at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
> >     at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
> >     at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
> >     at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
> >     at
> >
> >
> kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
> >     at
> >
> >
> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
> >     at
> >
> >
> kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:129)
> >     at kafka.tools.MirrorMaker$$anonfun$3.apply(MirrorMaker.scala:102)
> >     at kafka.tools.MirrorMaker$$anonfun$3.apply(MirrorMaker.scala:102)
> >     at
> >
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> >     at
> >
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> >     at
> >
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
> >     at scala.collection.immutable.List.foreach(List.scala:45)
> >     at
> > scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
> >     at scala.collection.immutable.List.map(List.scala:45)
> >     at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:102)
> >     at kafka.tools.MirrorMaker.main(MirrorMaker.scala)
> >
> > The command I run is
> >
> > $KAFKA_ROOT/bin/kafka-run-class.sh kafka.tools.MirrorMaker
> > --consumer.config ./mirror_consumer.properties --producer.config
> > ./mirror_producer.properties --whitelist=".*" --num.streams 1
> >
> > The ./mirror_consumer.properties file contains:
> >
> > broker.list=1:localhost:59092,2:localhost:59093
> > groupid=group1
> > shallowiterator.enable=true
> >
> > When broker.list is commented out and zk.connect is defined instead (or
> > when both are defined), mirroring happens successfully.
> >
> > Any leads?
> >
> > Thank you,
> > Aman
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message