kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewen Cheslack-Postava <e...@confluent.io>
Subject Re: Mirrormaker issue with Kafka 0.9 (confluent platform 2.0)
Date Thu, 10 Dec 2015 23:21:38 GMT
Meghana,

It looks like this functionality was removed in
https://issues.apache.org/jira/browse/KAFKA-1650, although I don't see
explicit discussion of the issue in the JIRA so I'm not sure of the exact
motivation. Maybe Becket or Guozhang can offer some insight about if it is
necessary (that JIRA fixes a data loss issue) or an accidental
incompatibility.

-Ewen

On Thu, Dec 10, 2015 at 2:32 PM, Meghana Narasimhan <
mnarasimhan@bandwidth.com> wrote:

> Hi,
>
> We upgraded to the Confluent 2.0 platform today (we were earlier using the
> Kafka 0.8 without the confluent platform). With the latest Kafka 0.9 that
> is packaged with the platform we are facing issues starting the Mirror
> maker with multiple consumer config files.
>
> The original command that we were using with kafka 0.8 was,
>
> bin/kafka-run-class kafka.tools.MirrorMaker  --consumer.config
> mirrorCluster1Consumer.properties --consumer.config
> mirrorCluster2Consumer.properties --consumer.config
> mirrorCluster3Consumer.properties --consumer.config
> mirrorCluster4Consumer.properties --producer.config
> mirrorProducer.properties --whitelist "topic1,topic2"
>
> With the current Kafka 0.9 release, the tool throws the following error
>
> ERROR Exception when starting mirror maker. (kafka.tools.MirrorMaker$)
> joptsimple.MultipleArgumentsForOptionException: Found multiple arguments
> for option ['consumer.config'], but you asked for only one
>         at joptsimple.OptionSet.valueOf(OptionSet.java:177)
>         at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:225)
>         at kafka.tools.MirrorMaker.main(MirrorMaker.scala)
> Exception in thread "main" java.lang.NullPointerException
>         at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:276)
>         at kafka.tools.MirrorMaker.main(MirrorMaker.scala)
>
> So I tried setting the --num.streams=4 option in the command, and it still
> throws the same error, next I tried to use the --new.consumer option but
> the tool fails again with the same error.
>
> Not sure how to get the Mirrormaker tool running with multiple consumer
> config files.
>
> Could not find much documentation regarding this except for the one in the
> kafka documentation (
> http://kafka.apache.org/documentation.html#basic_ops_mirror_maker).
>
> ps : I was unable to post this in the confluent mailing list and hence
> posting here.
>
> Thanks,
> Meghana
>



-- 
Thanks,
Ewen

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message