kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bhavesh Patel <bhavesh3...@gmail.com>
Subject Re: Kafka stream issue : Deleting obsolete state directory
Date Thu, 20 Sep 2018 18:31:35 GMT
Hi,
Is it possible to get h lp with below issue?

Many Thanks,
Bhavesh Patel

On Wed, Sep 19, 2018, 09:13 Bhavesh Patel <bhavesh3184@gmail.com> wrote:

> Hi Bill,
> Apologies, I was trying to attached image.
> Please find below the error log,
>
> 2018-09-18 09:26:09.112  INFO 1 --- [5-CleanupThread]
> o.a.k.s.p.internals.StateDirectory       : stream-thread
> [ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread]
> Deleting obsolete state directory 2_1 for task 2_1 as 601112ms has elapsed
> (cleanup delay is 600000ms).
>
> 2018-09-18 09:26:09.116 ERROR 1 --- [5-CleanupThread]
> o.a.k.s.p.internals.StateDirectory       : stream-thread
> [ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread] Failed
> to delete the state directory.
>
>
>
> java.nio.file.DirectoryNotEmptyException:
> /tmp/ApplicationName/ApplicationName/2_1
>
>                 at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
> ~[na:1.8.0_161]
>
>                 at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> ~[na:1.8.0_161]
>
>                 at java.nio.file.Files.delete(Files.java:1126)
> ~[na:1.8.0_161]
>
>                 at
> org.apache.kafka.common.utils.Utils$2.postVisitDirectory(Utils.java:740)
> ~[kafka-clients-2.0.0.jar!/:na]
>
>                 at
> org.apache.kafka.common.utils.Utils$2.postVisitDirectory(Utils.java:723)
> ~[kafka-clients-2.0.0.jar!/:na]
>
>                 at java.nio.file.Files.walkFileTree(Files.java:2688)
> ~[na:1.8.0_161]
>
>                 at java.nio.file.Files.walkFileTree(Files.java:2742)
> ~[na:1.8.0_161]
>
>                 at
> org.apache.kafka.common.utils.Utils.delete(Utils.java:723)
> ~[kafka-clients-2.0.0.jar!/:na]
>
>                 at
> org.apache.kafka.streams.processor.internals.StateDirectory.cleanRemovedTasks(StateDirectory.java:287)
> [kafka-streams-2.0.0.jar!/:na]
>
>                 at
> org.apache.kafka.streams.processor.internals.StateDirectory.cleanRemovedTasks(StateDirectory.java:250)
> [kafka-streams-2.0.0.jar!/:na]
>
>                 at
> org.apache.kafka.streams.KafkaStreams$2.run(KafkaStreams.java:800)
> [kafka-streams-2.0.0.jar!/:na]
>
>                 at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_161]
>
>                 at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_161]
>
>                 at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_161]
>
>                 at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_161]
>
>                 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
>
>                 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
>
>                 at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
>
> Many Thanks,
> Bhavesh Patel
>
>
> On Wed, Sep 19, 2018, 02:02 Bill Bejeck <bill@confluent.io> wrote:
>
>> Hi Bhavesh,
>>
>> Sorry, but I'm still not seeing any log file attachment, maybe it's being
>> filtered out.
>>
>> Can you copy the relevant section in the body of the email?
>>
>> Thanks,
>> Bill
>>
>> On Tue, Sep 18, 2018 at 3:21 PM Bhavesh Patel <bhavesh3184@gmail.com>
>> wrote:
>>
>> > Hi Bill,
>> >
>> > Apologies, Please find the error stack attached.
>> >
>> > We are using kafka 2.0.
>> >
>> > Thanks,
>> > Bhavesh Patel
>> >
>> > On Tue, Sep 18, 2018 at 8:04 PM Bill Bejeck <bill@confluent.io> wrote:
>> >
>> >> Hi Bhavesh,
>> >>
>> >> I don't see the log file attachment, can you resend it?
>> >>
>> >> Also, what version of Kafka Streams are you running?
>> >>
>> >> Thanks,
>> >> Bill
>> >>
>> >> On Tue, Sep 18, 2018 at 12:31 PM Bhavesh Patel <bhavesh3184@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > We have a stream application where we are facing a little strange
>> >> problem
>> >> > with deleting state directory.
>> >> >
>> >> > Our application is deployed into in-house container platform and
>> >> storage is
>> >> > backed by nfs.
>> >> >
>> >> > When app is running for a while and when it hits the cleanup delay
of
>> >> > 600000ms as defined dafault in Kafka settings we are getting
>> exception
>> >> of
>> >> > DIRECTORYNOTEMPTYEXCEPTION.
>> >> >
>> >> > I am attaching the log below, thoughts on this issue will be much
>> >> > appreciated.
>> >> >
>> >> > Many Thanks,
>> >> > Bhavesh Patel
>> >> >
>> >>
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message