kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: kafka-reassign-partitions.sh leaves data on source broker?
Date Sun, 01 Jun 2014 00:15:09 GMT
Hi Guozhang,

I don't have any replication-related properties in my config file and I
don't see any mentions of replication at startup in the log, so I don't
really know.

The only time I saw replication mentioned in the log was during some errors
that said my replication factor was 1 and that was > the number of brokers.
 So based on this error message I'd say my replication factor is 1.

Is there a replication property I could explicitly set?

Thanks,
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Sat, May 31, 2014 at 7:05 PM, Guozhang Wang <wangguoz@gmail.com> wrote:

> Hello Otis,
>
> What is your replication factor?
>
> Guozhang
>
>
> On Sat, May 31, 2014 at 12:39 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
> > Hi,
> >
> > I have 2 pretty unbalanced brokers running 0.8.1.1 and I'm trying to
> > balance them (read: free disk space on the overloaded broker).
> >
> > Is there a tool that will just rebalance everything?  I couldn't find
> one,
> > so I ran this:
> >
> > $ sh kafka-reassign-partitions.sh --broker-list 1 --zookeeper
> > zk1.prod.sematext:3131 --reassignment-json-file /tmp/partition.json
> > --execute
> >
> > This seems to have *copied* about half of the partitions of the topic
> > specified in the given JSON file from broker 2 (source) to broker 1
> > (target). Good.
> >
> > I see this in broker 2 (source broker):
> >
> > [2014-05-31 19:24:59,994] INFO [ReplicaFetcherManager on broker 2]
> Removed
> > fetcher for partitions [spm_new_cluster_free_system_topic,1]
> > (kafka.server.ReplicaFetcherManager)
> >
> > ... which looks promising.... but:
> >
> > My broker 2 (source) *still* has all partitions of the problematic topic.
> > I thought it would remove the moved partitions from the source broker....
> > no?
> >
> > Thanks,
> > Otis
> > --
> > Performance Monitoring * Log Analytics * Search Analytics
> > Solr & Elasticsearch Support * http://sematext.com/
> >
>
>
>
> --
> -- Guozhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message