kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apurva Mehta <apu...@confluent.io>
Subject Re: lag for a specific partitions on newly added host
Date Wed, 14 Dec 2016 19:46:28 GMT
Which version of Kafka are you using? How did you run the tool exactly? Can
you share the command line?

On Tue, Dec 13, 2016 at 6:05 PM, Jeremy Hansen <jeremy@skidrow.la> wrote:

> The new host has been in place for over a week. Lag is still high on
> partitions that exist on that new host.  Should I attempt another reassign?
>
> Thanks
> -jeremy
>
> > On Dec 13, 2016, at 5:43 PM, Apurva Mehta <apurva@confluent.io> wrote:
> >
> > How did you add the host and when did you measure the lag? If you used
> the
> > reassign-partitions script, it will move partitions to the new host, but
> > the data copy will take time. in that period, those partitions will lag.
> > However, once the reassign-partitions script finishes, the partitions on
> > the new replica should be caught up and no longer demonstrate any lag.
> >
> >> On Fri, Dec 9, 2016 at 8:58 PM, Jeremy Hansen <jeremy@skidrow.la>
> wrote:
> >>
> >> Here’s the topic description:
> >>
> >> Topic:blumfrub  PartitionCount:15       ReplicationFactor:5     Configs:
> >>        Topic: blumfrub Partition: 0    Leader: 1001    Replicas:
> >> 1001,0,1,2,3  Isr: 0,1001,1,2,3
> >>        Topic: blumfrub Partition: 1    Leader: 0       Replicas:
> >> 0,1,2,3,4     Isr: 0,1,2,3,4
> >>        Topic: blumfrub Partition: 2    Leader: 1       Replicas:
> >> 1,2,3,4,1001  Isr: 1001,1,2,3,4
> >>        Topic: blumfrub Partition: 3    Leader: 2       Replicas:
> >> 2,3,4,1001,0  Isr: 0,1001,2,3,4
> >>        Topic: blumfrub Partition: 4    Leader: 0       Replicas:
> >> 3,4,1001,0,1  Isr: 0,1001,1,3,4
> >>        Topic: blumfrub Partition: 5    Leader: 4       Replicas:
> >> 4,1001,0,1,2  Isr: 0,1001,1,2,4
> >>        Topic: blumfrub Partition: 6    Leader: 1001    Replicas:
> >> 1001,1,2,3,4  Isr: 1001,1,2,3,4
> >>        Topic: blumfrub Partition: 7    Leader: 0       Replicas:
> >> 0,2,3,4,1001  Isr: 0,1001,2,3,4
> >>        Topic: blumfrub Partition: 8    Leader: 1       Replicas:
> >> 1,3,4,1001,0  Isr: 0,1001,1,3,4
> >>        Topic: blumfrub Partition: 9    Leader: 2       Replicas:
> >> 2,4,1001,0,1  Isr: 0,1001,1,2,4
> >>        Topic: blumfrub Partition: 10   Leader: 0       Replicas:
> >> 3,1001,0,1,2  Isr: 0,1001,1,2,3
> >>        Topic: blumfrub Partition: 11   Leader: 4       Replicas:
> >> 4,0,1,2,3     Isr: 0,1,2,3,4
> >>        Topic: blumfrub Partition: 12   Leader: 1001    Replicas:
> >> 1001,2,3,4,0  Isr: 0,1001,2,3,4
> >>        Topic: blumfrub Partition: 13   Leader: 0       Replicas:
> >> 0,3,4,1001,1  Isr: 0,1001,1,3,4
> >>        Topic: blumfrub Partition: 14   Leader: 1       Replicas:
> >> 1,4,1001,0,2  Isr: 0,1001,1,2,4
> >>
> >> 1001 is the new broker.
> >>
> >> -jeremy
> >>
> >>
> >>
> >>> On Dec 9, 2016, at 8:55 PM, Jeremy Hansen <jeremy@skidrow.la> wrote:
> >>>
> >>> I added a new host to kafka.  Partitions that fall on this new host
> have
> >> a very high lag and I’m trying to understand why this would be and how
> to
> >> fix it.
> >>>
> >>> iloveconsuming                      blumfrub              0
> >> 5434682         7416933         1982251
>  iloveconsuming_kf0001.host.
> >> com-1481225033576-47393b55-0
> >>> iloveconsuming                      blumfrub              1
> >> 7416828         7416875         47
> iloveconsuming_kf0001.host.
> >> com-1481225033769-17152bca-0
> >>> iloveconsuming                      blumfrub              2
> >> 7416799         7416848         49
> iloveconsuming_kf0001.host.
> >> com-1481225033791-77a30285-0
> >>> iloveconsuming                      blumfrub              3
> >> 7416898         7416903         5
>  iloveconsuming_kf0001.host.
> >> com-1481225033822-d088f844-0
> >>> iloveconsuming                      blumfrub              4
> >> 7416891         7416925         34
> iloveconsuming_kf0001.host.
> >> com-1481225033846-78f8e5b5-0
> >>> iloveconsuming                      blumfrub              5
> >> 7416843         7416883         40
> iloveconsuming_kf0001.host.
> >> com-1481225033869-54027178-0
> >>> iloveconsuming                      blumfrub              6
> >> 5434720         7416896         1982176
>  iloveconsuming_kf0001.host.
> >> com-1481225033891-cc3f6bf6-0
> >>> iloveconsuming                      blumfrub              7
> >> 7416896         7416954         58
> iloveconsuming_kf0001.host.
> >> com-1481225033915-79b49de8-0
> >>> iloveconsuming                      blumfrub              8
> >> 7416849         7416898         49
> iloveconsuming_kf0001.host.
> >> com-1481225033939-1fe784c0-0
> >>> iloveconsuming                      blumfrub              9
> >> 7416898         7416917         19
> iloveconsuming_kf0001.host.
> >> com-1481225033961-40cc3185-0
> >>> iloveconsuming                      blumfrub              10
> >> 35457186        35457221        35
> iloveconsuming_kf0001.host.
> >> com-1481225033998-a817062e-0
> >>> iloveconsuming                      blumfrub              11
> >> 7416866         7416909         43
> iloveconsuming_kf0001.host.
> >> com-1481225034020-7a15999e-0
> >>> iloveconsuming                      blumfrub              12
> >> 5434739         7416907         1982168
>  iloveconsuming_kf0001.host.
> >> com-1481225034043-badde97c-0
> >>> iloveconsuming                      blumfrub              13
> >> 7416818         7416865         47
> iloveconsuming_kf0001.host.
> >> com-1481225034066-6e77e3dc-0
> >>> iloveconsuming                      blumfrub              14
> >> 7416901         7416947         46
> iloveconsuming_kf0002.host.
> >> com-1481225107317-32355c51-0
> >>>
> >>> Why do the partitions that fall on the newly added host so lagged?
> >>>
> >>> Thanks
> >>> -jeremy
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message