kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhavesh Patel <bhavesh3...@gmail.com>
Subject Re: Kafka stream issue : Deleting obsolete state directory
Date Wed, 19 Sep 2018 08:13:10 GMT
Hi Bill,
Apologies, I was trying to attached image.
Please find below the error log,

2018-09-18 09:26:09.112  INFO 1 --- [5-CleanupThread]
o.a.k.s.p.internals.StateDirectory       : stream-thread
[ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread]
Deleting obsolete state directory 2_1 for task 2_1 as 601112ms has elapsed
(cleanup delay is 600000ms).

2018-09-18 09:26:09.116 ERROR 1 --- [5-CleanupThread]
o.a.k.s.p.internals.StateDirectory       : stream-thread
[ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread] Failed
to delete the state directory.



java.nio.file.DirectoryNotEmptyException:
/tmp/ApplicationName/ApplicationName/2_1

                at
sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
~[na:1.8.0_161]

                at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
~[na:1.8.0_161]

                at java.nio.file.Files.delete(Files.java:1126)
~[na:1.8.0_161]

                at
org.apache.kafka.common.utils.Utils$2.postVisitDirectory(Utils.java:740)
~[kafka-clients-2.0.0.jar!/:na]

                at
org.apache.kafka.common.utils.Utils$2.postVisitDirectory(Utils.java:723)
~[kafka-clients-2.0.0.jar!/:na]

                at java.nio.file.Files.walkFileTree(Files.java:2688)
~[na:1.8.0_161]

                at java.nio.file.Files.walkFileTree(Files.java:2742)
~[na:1.8.0_161]

                at
org.apache.kafka.common.utils.Utils.delete(Utils.java:723)
~[kafka-clients-2.0.0.jar!/:na]

                at
org.apache.kafka.streams.processor.internals.StateDirectory.cleanRemovedTasks(StateDirectory.java:287)
[kafka-streams-2.0.0.jar!/:na]

                at
org.apache.kafka.streams.processor.internals.StateDirectory.cleanRemovedTasks(StateDirectory.java:250)
[kafka-streams-2.0.0.jar!/:na]

                at
org.apache.kafka.streams.KafkaStreams$2.run(KafkaStreams.java:800)
[kafka-streams-2.0.0.jar!/:na]

                at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_161]

                at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_161]

                at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_161]

                at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_161]

                at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_161]

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_161]

                at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]

Many Thanks,
Bhavesh Patel


On Wed, Sep 19, 2018, 02:02 Bill Bejeck <bill@confluent.io> wrote:

> Hi Bhavesh,
>
> Sorry, but I'm still not seeing any log file attachment, maybe it's being
> filtered out.
>
> Can you copy the relevant section in the body of the email?
>
> Thanks,
> Bill
>
> On Tue, Sep 18, 2018 at 3:21 PM Bhavesh Patel <bhavesh3184@gmail.com>
> wrote:
>
> > Hi Bill,
> >
> > Apologies, Please find the error stack attached.
> >
> > We are using kafka 2.0.
> >
> > Thanks,
> > Bhavesh Patel
> >
> > On Tue, Sep 18, 2018 at 8:04 PM Bill Bejeck <bill@confluent.io> wrote:
> >
> >> Hi Bhavesh,
> >>
> >> I don't see the log file attachment, can you resend it?
> >>
> >> Also, what version of Kafka Streams are you running?
> >>
> >> Thanks,
> >> Bill
> >>
> >> On Tue, Sep 18, 2018 at 12:31 PM Bhavesh Patel <bhavesh3184@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > We have a stream application where we are facing a little strange
> >> problem
> >> > with deleting state directory.
> >> >
> >> > Our application is deployed into in-house container platform and
> >> storage is
> >> > backed by nfs.
> >> >
> >> > When app is running for a while and when it hits the cleanup delay of
> >> > 600000ms as defined dafault in Kafka settings we are getting exception
> >> of
> >> > DIRECTORYNOTEMPTYEXCEPTION.
> >> >
> >> > I am attaching the log below, thoughts on this issue will be much
> >> > appreciated.
> >> >
> >> > Many Thanks,
> >> > Bhavesh Patel
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message