kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Field <jvfi...@blizzard.com>
Subject Re: Kafka Mirrormaker issue
Date Fri, 06 Apr 2018 20:09:08 GMT
I'm hitting the same problem, even with the new consumer, on MirrorMaker 0.9 reading from a
0.9 Kafka cluster and producing to a 0.11 Kafka cluster.

On 3/30/18, 3:56 PM, "Andrew Otto" <otto@wikimedia.org> wrote:

    I’m currently stuck on MirrorMaker version 0.9, and I’m not sure when the
    new consumer client became the default.  Does your 0.10 version have a
    —new.consumer option listed in the help message?  If so, then the new
    consumer client is not the default.  I haven’t seen the problem you are
    describing (I’m still having plenty of others though) since I’ve switched
    to using the new consumer.
    
    Another thought, what is the value of your partition.assignment.strategy?
    I’ve found round robin (default in later versions of MirrorMaker) to be a
    lot more consistent than whatever the default is in 0.9.  Not sure what the
    default in 0.10 is.
    
    
    
    On Fri, Mar 30, 2018 at 11:40 AM, Siva A <siva9940261121@gmail.com> wrote:
    
    > Any other update on this?
    >
    > On Mon, Mar 26, 2018, 7:42 PM Andrew Otto <otto@wikimedia.org> wrote:
    >
    > > I’ve had similar problems, but I don’t have an explanation for ya :/
    > >
    > > On Sun, Mar 25, 2018 at 12:19 PM, Siva A <siva9940261121@gmail.com>
    > wrote:
    > >
    > > > Hi,
    > > >
    > > > We have 3 nodes Kafka cluster(0.10.0.1) and its mirroring the data from
    > > > another 3 node cluster of same Kafka version.
    > > > Both the clusters are Kerberized and we are running the Mirrormaker on
    > > the
    > > > target cluster using the single principal/keytab with the one way trust
    > > on
    > > > the KDC.
    > > >
    > > > At times, the mirrormaker stops functioning(Doesn't mirror the data)
    > but
    > > > the process is still running. If we restart the service then it works
    > > fine
    > > > for a day or so.
    > > >
    > > > I don't see any error on the Kafka logs as well.
    > > > Is there anyone seen this kind of issue?
    > > >
    > > > Thanks
    > > > Siva
    > > >
    > >
    >
    

Mime
View raw message