nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Gilman <matt.c.gil...@gmail.com>
Subject Re: Queue incoherent state
Date Tue, 16 May 2017 12:15:26 GMT
Thanks for following up with these details. I'm going to add these
observations to the corresponding JIRA [1].

Thanks and sorry again for the inconvenience.

Matt

[1] https://issues.apache.org/jira/browse/NIFI-3897

On Tue, May 16, 2017 at 3:34 AM, Arnaud G <greatpatton@gmail.com> wrote:

> Hi Matt,
>
> Thanks for your reply!
>
> I finally solved the problem by deleting all the content in the flowfile
> directory, but here are my observations:
>
> 1) The problem was coming from one of the cluster node, when this node was
> out of the cluster, the queue were reporting 0 flowfile.
> 2) The first time I restarted this node, about 20'000 flowfile reappeared
> and were treated, every time I subsequently restarted this node about
> 20-30k flowfiles were again treated (I was only specifically monitoring one
> queue, but it happened for multiple other queues)
> 3) After 3-4 reboots of this node the queue reported 90K elements and
> remained in this state despite multiple other restart.
> 4) The flowfile directory on this node contained 200 MB of data
> 5) I tried to setup the flowfile expiration but it didn't do anything to
> the queue status
> 6) I tried to change the backpressure threshold without any effect.
> 7) During the problem the queue was operating normally on the cluster, and
> flowfiles were flowing through it without any issue.
>
> Arnaud
>
>
>
> On Mon, May 15, 2017 at 10:39 PM, Matt Gilman <matt.c.gilman@gmail.com>
> wrote:
>
>> Sorry for the delayed response. Similar behavior has been reported by
>> some other users [1]. Does the connection have any back pressure threshold
>> configured? Can new flowfiles be enqueued? Do the expiration settings have
>> any affect?
>>
>> Lastly, if you restart the cluster does it claim the connection still has
>> flowfiles enqueued?
>>
>> Matt
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-3897
>>
>> On Fri, May 12, 2017 at 5:47 AM, Arnaud G <greatpatton@gmail.com> wrote:
>>
>>> Hi again!
>>>
>>> I currently have  another issue with incoherent queue status.
>>>
>>> Following the upgrade to 1.2 of a cluster, I have a couple of queues
>>> that display through the GUI a high number of flowfiles.
>>>
>>> As the queue were no emptying despite tuning, I tried to list the
>>> content of the queue. This action returns that the queue contains no
>>> flowfile, which is not the expected as the GUI displays another value.
>>>
>>> If I try to empty the queue, I receive a message: 0 FlowFiles (0 bytes)
>>> out of 210'000 (92.71MB) were removed from the queue.
>>>
>>> And of course I cannot delete the queue as this action reports me that
>>> the queue is not empty.
>>>
>>> So somehow it seems that the queue are empty but that the current
>>> display of the queue don't reflect it (it is very likely that some data
>>> were lost during the upgrade procedure as we had to reboot a few node to
>>> change the heap property)
>>>
>>> What will be the best method to restore a proper state and be able to
>>> edit the flow file again?
>>>
>>> Thank you!
>>>
>>> Arnaud
>>>
>>>
>>>
>>
>

Mime
View raw message