nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: Apache Nifi issues
Date Thu, 09 May 2019 16:10:59 GMT
Suman

The community resolved these (and many other) issues in Apache NiFi 1.9.x
line.

There are vendor offerings that have addressed it similarly but which may
be more direct to the line you're talking about.

Thanks

On Thu, May 9, 2019 at 11:43 AM Suman B N <sumannewton@gmail.com> wrote:

> Any turn around solutions in version 1.8 to resolve these? If not then I
> will plan to upgrade soon. If there are any quicker solutions which can be
> applied in 1.8 without any data loss then I ll upgrade it later.
>
> On Thursday, May 9, 2019, Joe Witt <joe.witt@gmail.com> wrote:
>
>> Suman
>>
>> Yeah it looks related to the queue/load balance fixes.  Latest release
>> should be much better for you.
>>
>> Thanks
>>
>>
>> On Thu, May 9, 2019, 5:30 AM Suman B N <sumannewton@gmail.com> wrote:
>>
>>> Team,
>>> We are running a 3 node nifi cluster in docker. Version is 1.8.
>>> Everything has been running smoothly from the last 2-3 months. But from
>>> last 2 days, we are observing a couple of issues and those are explained
>>> below:
>>>
>>>
>>>    - *Load Balanced Client Thread throwing exceptions continuously.*
>>>    Nifi logs are filled with below errors and making the disk full very soon.
>>>
>>> Stacktrace:
>>>
>>> 2019-05-09 09:06:26,166 ERROR [Load-Balanced Client Thread-1]
>>> o.a.n.c.queue.SwappablePriorityQueue Updated Size of Queue Unacknowledged
>>> from FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0
>>> Bytes], Swap Files=[0], Unacknowledged=[-484, -50212519 Bytes] ] to
>>> FlowFile Queue Size[ ActiveQueue=[0, 0 Bytes], Swap Queue=[0, 0 Bytes],
>>> Swap Files=[0], Unacknowledged=[-484, -50212519 Bytes] ]
>>>
>>> java.lang.RuntimeException: Cannot create negative queue size
>>>
>>> at
>>> org.apache.nifi.controller.queue.SwappablePriorityQueue.logIfNegative(SwappablePriorityQueue.java:925)
>>>
>>> at
>>> org.apache.nifi.controller.queue.SwappablePriorityQueue.incrementUnacknowledgedQueueSize(SwappablePriorityQueue.java:915)
>>>
>>> at
>>> org.apache.nifi.controller.queue.SwappablePriorityQueue.acknowledge(SwappablePriorityQueue.java:417)
>>>
>>> at
>>> org.apache.nifi.controller.queue.clustered.partition.RemoteQueuePartition$2.onTransactionComplete(RemoteQueuePartition.java:210)
>>>
>>> at
>>> org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClient.communicate(NioAsyncLoadBalanceClient.java:259)
>>>
>>> at
>>> org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClientTask.run(NioAsyncLoadBalanceClientTask.java:76)
>>>
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>
>>> at java.lang.Thread.run(Thread.java:748)
>>>
>>>    - *Flow files are stuck in queue.* Probably its this issue -
>>>    NIFI-5919 <https://issues.apache.org/jira/browse/NIFI-5919> which is
>>>    fixed in 1.9.
>>>
>>> The flow files are stuck in the queue(Load balance by attribute) and are
>>> not read by the next downstream processor(MergeRecord with CSVReader and
>>> CSVRecordSetWriter). From the Nifi UI, it appears that flow files are in
>>> the queue but when tried to list queue it says "Queue has no flow files".
>>> Attempting to empty queue also gives the exact message. I have tried below
>>> action items but all in vain:
>>>
>>>    - Restarting the downstream and upstream(ConvertRecord) processor.
>>>       - Disabled and enabled CSVReader and CSVRecordSetWriter.
>>>       - Disabled load balancing.
>>>
>>> How can I debug this or resolve this? I am pretty new to nifi. Trying my
>>> best to understand the flow file life cycle and internal architecture of
>>> nifi.
>>>
>>> Any leads or assistance to solve the above two issues is very much
>>> appreciated.
>>> --
>>> *Suman*
>>> *Tathastu*
>>>
>>
>
> --
> *Suman*
> *OlaCabs*
>
>

Mime
View raw message