kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liam Clarke <liam.cla...@adscale.co.nz>
Subject Re: Configuration of log compaction
Date Tue, 18 Dec 2018 22:05:17 GMT
Kia ora Claudia,

What version Kafka are you running? I can't find the specified code in
Kafka 1.0.0.

With regards to your questions:

1) I don't think so, but I'm just guessing here.
2) The log cleaner logs, if piped into something like Elasticsearch could
be used to monitor it?
3) No, it's a thread started by the broker. I imagine you're using an old
broker version, around 0.8.2 perhaps? We've found that rolling broker
restarts with 0.11 are rather easy and not to be feared.

Kind regards,

Liam Clarke

On Tue, Dec 18, 2018 at 10:43 PM Claudia Wegmann <c.wegmann@kasasi.de>
wrote:

> Hi Liam,
>
> thanks for the pointer. I found out, that the log cleaner on all kafka
> brokers died with the following error message:
>
> [2018-12-04 15:33:24,886] INFO Cleaner 0: Caught segment overflow error
> during cleaning: Detected offset overflow at offset -1 in segment
> LogSegment(baseOffset=3605669, size=6970326) (kafka.log.LogCleaner)
> [2018-12-04 15:33:24,958] ERROR [kafka-log-cleaner-thread-0]: Error due to
> (kafka.log.LogCleaner)
> java.lang.IllegalArgumentException: requirement failed: Split operation is
> only permitted for segments with overflow
>         at scala.Predef$.require(Predef.scala:277)
>         at kafka.log.Log.splitOverflowedSegment(Log.scala:1873)
>         at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:517)
>         at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:465)
>         at
> kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:464)
>         at scala.collection.immutable.List.foreach(List.scala:389)
>         at kafka.log.Cleaner.doClean(LogCleaner.scala:464)
>         at kafka.log.Cleaner.clean(LogCleaner.scala:442)
>         at
> kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:303)
>         at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:289)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
> [2018-12-04 15:33:24,959] INFO [kafka-log-cleaner-thread-0]: Stopped
> (kafka.log.LogCleaner)
>
> This brought up some new questions:
> 1.) How do I avoid this error? Did I misconfigure the creation of segments?
> 2.) Is there a way to monitor the log cleaner?
> 3.) Can I restart the log cleaner manually without restarting the whole
> broker?
>
> Thanks for your help.
>
> Best,
> Claudia
>
> -----Urspr√ľngliche Nachricht-----
> Von: Liam Clarke <liam.clarke@adscale.co.nz>
> Gesendet: Montag, 17. Dezember 2018 23:06
> An: users@kafka.apache.org
> Betreff: Re: Configuration of log compaction
>
> Hi Claudia,
>
> Anything useful in the log cleaner log files?
>
> Cheers,
>
> Liam Clarke
>
> On Tue, 18 Dec. 2018, 3:18 am Claudia Wegmann <c.wegmann@kasasi.de wrote:
>
> > Hi,
> >
> > thanks for the quick response.
> >
> > My problem is not, that no new segments are created, but that segments
> > with old data do not get compacted.
> > I had to restart one broker because there was no diskspace left. After
> > recreating all indexes etc. the broker recognized the old data and
> > compacted it correctly. I had to restart all other brokers of the
> > cluster, too, for them to also recognize the old data and start
> compacting.
> >
> > So I guess, before restarting the brokers where to busy to
> > compact/delete old data? Is there a configuration to ensure compaction
> > after a certain amount of time or something?
> >
> > Best,
> > Claudia
> >
> > -----Urspr√ľngliche Nachricht-----
> > Von: Spico Florin <spicoflorin@gmail.com>
> > Gesendet: Montag, 17. Dezember 2018 14:28
> > An: users@kafka.apache.org
> > Betreff: Re: Configuration of log compaction
> >
> > Hello!
> >   Please check whether the segment.ms configuration on topic will help
> > you to solve your problem.
> >
> > https://kafka.apache.org/documentation/
> >
> >
> > https://stackoverflow.com/questions/41048041/kafka-deletes-segments-ev
> > en-before-segment-size-is-reached
> >
> > Regards,
> >  Florin
> >
> > segment.ms This configuration controls the period of time after which
> > Kafka will force the log to roll even if the segment file isn't full
> > to ensure that retention can delete or compact old data. long
> > 604800000 [1,...] log.roll.ms medium
> >
> > On Mon, Dec 17, 2018 at 12:28 PM Claudia Wegmann <c.wegmann@kasasi.de>
> > wrote:
> >
> > > Dear kafka users,
> > >
> > > I've got a problem on one of my kafka clusters. I use this cluster
> > > with kafka streams applications. Some of this stream apps use a
> > > kafka state store. Therefore a changelog topic is created for those
> > > stores with cleanup policy "compact". One of these topics is running
> > > wild for some time now and seems to grow indefinitely. When I check
> > > the  log file of the first segment, there is a lot of data in it,
> > > that should have been compacted already.
> > >
> > > So I guess I did not configure everything correctly for log
> > > compaction to work as expected. What config parameters do have
> > > influence on log compaction? And how to set them, when I want data
> > > older than 4 hours to be compacted?
> > >
> > > Thanks in advance.
> > >
> > > Best,
> > > Claudia
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message