kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Crayford <tcrayf...@heroku.com>
Subject Re: offset topics growing huge
Date Fri, 07 Oct 2016 12:55:13 GMT
On Mon, Oct 3, 2016 at 5:38 PM, Ali Akhtar <ali.rac200@gmail.com> wrote:

> Newbie question, but what exactly does log.cleaner.enable=true do, and how
> do I know if I need to set it to be true?
>

If you're using any compacted topics (including __consumer_offsets), it
needs to be on.


>
> Also, if config changes like that need to be made once a cluster is up and
> running, what's the recommended way to do that? Do you killall -12 kafka
> and then make the change, and then start kafka again, one broker at a time?
>

Roughly yes. You should probably orchestrate it and run e.g. health checks
and such before restarting the next node.

Thanks

Tom Crayford
Heroku Kafka


>
> On Mon, Oct 3, 2016 at 9:27 PM, Tom Crayford <tcrayford@heroku.com> wrote:
>
> > Yes, offset topic compaction is just the normal compaction.
> >
> > Thanks
> >
> > Tom Crayford
> > Heroku Kafka
> >
> > On Monday, 3 October 2016, Tobias Adamson <tobias@stargazer.com.sg>
> wrote:
> >
> > > Hi
> > > We are using Kafka 0.10.1 with offsets commits being stored inside of
> > Kafka
> > > After a while these topics become extremely large and we are wondering
> if
> > > we
> > > need to enable log.cleaner.enable=true (currently false) to make sure
> the
> > > internal
> > > offset topics get compacted and keep their size down?
> > >
> > > Regards
> > > T
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message