nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: MergeContent error on restart - not the most recent flow file in this session
Date Wed, 11 Jan 2017 20:28:02 GMT
Hi Ben,

I've not seen this issue, but I created a JIRA [1] to address it. Will try to replicate it
locally and see
if I can understand what is happening. When this happens, do you see the processor continually
spewing this Exception, rolling back, and trying again?

This error itself would not result in data loss. However, clearing the queue certainly would.

Thanks
-Mark


[1] https://issues.apache.org/jira/browse/NIFI-3329




On Jan 11, 2017, at 1:21 PM, bmichaud <ben_michaud@optum.com<mailto:ben_michaud@optum.com>>
wrote:

This error occurred in NiFi 1.1.1 (single node) right after an upgrade from
1.1.0, so I thought it was due to the upgrade, but now I see that it happens
if I restart as well and there is data in the flow.

This MergeContent processor is sorting incoming FlowFiles into bins based on
a correlation attribute and dumping out the accumulated uber-FlowFile after
one hour or 500K messages are accumulated.

Questions:
(1) Should I clear out all the flow files from the flow (terminate intake
and let everything drain out)?
(2) Will data be lost when this error occurs?

Error:
-------
2017-01-11 10:53:39,561 WARN [Timer-Driven Process Thread-9]
o.a.n.processors.standard.MergeContent
MergeContent[id=b9898788-1c11-45d9-8646-44b1dceb92d5] Processor
Administratively Yielded for 1 sec due to processing failure
2017-01-11 10:53:39,561 WARN [Timer-Driven Process Thread-9]
o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
MergeContent[id=b9898788-1c11-45d9-8646-44b1dceb92d5] due to uncaught
Exception: org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=07f3d75a-3347-45da-8dd4-fecf775d1dcf,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1484151577298-1971,
container=default, section=947], offset=566526,
length=64],offset=0,name=8551433629757608.tsv,size=64] is not the most
recent version of this FlowFile within this session
(StandardProcessSession[id=133767])
2017-01-11 10:53:39,563 WARN [Timer-Driven Process Thread-9]
o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=07f3d75a-3347-45da-8dd4-fecf775d1dcf,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1484151577298-1971,
container=default, section=947], offset=566526,
length=64],offset=0,name=8551433629757608.tsv,size=64] is not the most
recent version of this FlowFile within this session
(StandardProcessSession[id=133767])
       at
org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1121)
~[nifi-framework-core-1.1.0.jar:1.1.0]
       at
org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1102)
~[nifi-framework-core-1.1.0.jar:1.1.0]
       at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142)
~[na:na]
       at
org.apache.nifi.processor.util.bin.BinManager.offer(BinManager.java:194)
~[na:na]
       at
org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:279)
~[na:na]
       at
org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178)
~[na:na]
       at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
~[nifi-framework-core-1.1.0.jar:1.1.0]
       at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.1.0.jar:1.1.0]
       at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.1.0.jar:1.1.0]
       at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.1.0.jar:1.1.0]
       at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_65]
       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_65]
       at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_65]
       at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_65]
       at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_65]
       at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_65]
       at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]




--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/MergeContent-error-on-restart-not-the-most-recent-flow-file-in-this-session-tp14436.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com<http://Nabble.com>.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message