kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Bejeck <b...@confluent.io>
Subject Re: Kafka stream issue : Deleting obsolete state directory
Date Thu, 20 Sep 2018 20:30:43 GMT
Hi Bhavesh,

I haven't forgotten about you, just got sidetracked with some other
issues.

Just to be clear you are using Kafka Streams version 2.0 correct?

I'll take a look in the next day or so.

Thanks,
Bill

On Thu, Sep 20, 2018 at 2:31 PM Bhavesh Patel <bhavesh3184@gmail.com> wrote:

> Hi,
> Is it possible to get h lp with below issue?
>
> Many Thanks,
> Bhavesh Patel
>
> On Wed, Sep 19, 2018, 09:13 Bhavesh Patel <bhavesh3184@gmail.com> wrote:
>
> > Hi Bill,
> > Apologies, I was trying to attached image.
> > Please find below the error log,
> >
> > 2018-09-18 09:26:09.112  INFO 1 --- [5-CleanupThread]
> > o.a.k.s.p.internals.StateDirectory       : stream-thread
> > [ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread]
> > Deleting obsolete state directory 2_1 for task 2_1 as 601112ms has
> elapsed
> > (cleanup delay is 600000ms).
> >
> > 2018-09-18 09:26:09.116 ERROR 1 --- [5-CleanupThread]
> > o.a.k.s.p.internals.StateDirectory       : stream-thread
> > [ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread]
> Failed
> > to delete the state directory.
> >
> >
> >
> > java.nio.file.DirectoryNotEmptyException:
> > /tmp/ApplicationName/ApplicationName/2_1
> >
> >                 at
> >
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
> > ~[na:1.8.0_161]
> >
> >                 at
> >
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> > ~[na:1.8.0_161]
> >
> >                 at java.nio.file.Files.delete(Files.java:1126)
> > ~[na:1.8.0_161]
> >
> >                 at
> > org.apache.kafka.common.utils.Utils$2.postVisitDirectory(Utils.java:740)
> > ~[kafka-clients-2.0.0.jar!/:na]
> >
> >                 at
> > org.apache.kafka.common.utils.Utils$2.postVisitDirectory(Utils.java:723)
> > ~[kafka-clients-2.0.0.jar!/:na]
> >
> >                 at java.nio.file.Files.walkFileTree(Files.java:2688)
> > ~[na:1.8.0_161]
> >
> >                 at java.nio.file.Files.walkFileTree(Files.java:2742)
> > ~[na:1.8.0_161]
> >
> >                 at
> > org.apache.kafka.common.utils.Utils.delete(Utils.java:723)
> > ~[kafka-clients-2.0.0.jar!/:na]
> >
> >                 at
> >
> org.apache.kafka.streams.processor.internals.StateDirectory.cleanRemovedTasks(StateDirectory.java:287)
> > [kafka-streams-2.0.0.jar!/:na]
> >
> >                 at
> >
> org.apache.kafka.streams.processor.internals.StateDirectory.cleanRemovedTasks(StateDirectory.java:250)
> > [kafka-streams-2.0.0.jar!/:na]
> >
> >                 at
> > org.apache.kafka.streams.KafkaStreams$2.run(KafkaStreams.java:800)
> > [kafka-streams-2.0.0.jar!/:na]
> >
> >                 at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > [na:1.8.0_161]
> >
> >                 at
> > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > [na:1.8.0_161]
> >
> >                 at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > [na:1.8.0_161]
> >
> >                 at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > [na:1.8.0_161]
> >
> >                 at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [na:1.8.0_161]
> >
> >                 at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [na:1.8.0_161]
> >
> >                 at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> >
> > Many Thanks,
> > Bhavesh Patel
> >
> >
> > On Wed, Sep 19, 2018, 02:02 Bill Bejeck <bill@confluent.io> wrote:
> >
> >> Hi Bhavesh,
> >>
> >> Sorry, but I'm still not seeing any log file attachment, maybe it's
> being
> >> filtered out.
> >>
> >> Can you copy the relevant section in the body of the email?
> >>
> >> Thanks,
> >> Bill
> >>
> >> On Tue, Sep 18, 2018 at 3:21 PM Bhavesh Patel <bhavesh3184@gmail.com>
> >> wrote:
> >>
> >> > Hi Bill,
> >> >
> >> > Apologies, Please find the error stack attached.
> >> >
> >> > We are using kafka 2.0.
> >> >
> >> > Thanks,
> >> > Bhavesh Patel
> >> >
> >> > On Tue, Sep 18, 2018 at 8:04 PM Bill Bejeck <bill@confluent.io>
> wrote:
> >> >
> >> >> Hi Bhavesh,
> >> >>
> >> >> I don't see the log file attachment, can you resend it?
> >> >>
> >> >> Also, what version of Kafka Streams are you running?
> >> >>
> >> >> Thanks,
> >> >> Bill
> >> >>
> >> >> On Tue, Sep 18, 2018 at 12:31 PM Bhavesh Patel <
> bhavesh3184@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi,
> >> >> >
> >> >> > We have a stream application where we are facing a little strange
> >> >> problem
> >> >> > with deleting state directory.
> >> >> >
> >> >> > Our application is deployed into in-house container platform and
> >> >> storage is
> >> >> > backed by nfs.
> >> >> >
> >> >> > When app is running for a while and when it hits the cleanup delay
> of
> >> >> > 600000ms as defined dafault in Kafka settings we are getting
> >> exception
> >> >> of
> >> >> > DIRECTORYNOTEMPTYEXCEPTION.
> >> >> >
> >> >> > I am attaching the log below, thoughts on this issue will be much
> >> >> > appreciated.
> >> >> >
> >> >> > Many Thanks,
> >> >> > Bhavesh Patel
> >> >> >
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message