nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Conrad Crampton <conrad.cramp...@SecData.com>
Subject Re: NPE MergeContent processor
Date Thu, 10 Nov 2016 15:06:48 GMT
Hi,
The processor continues to write (to HDFS – the next processor in flow) and doesn’t block
any others coming into this processor (MergeContent), so not quite the same observed behaviour
as NIFI-2015.
If there is anything else you would like me to do to help with this more than happy to help.
Regards
Conrad

From: Bryan Bende <bbende@gmail.com>
Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
Date: Thursday, 10 November 2016 at 14:59
To: "users@nifi.apache.org" <users@nifi.apache.org>
Subject: Re: NPE MergeContent processor

Conrad,

Thanks for reporting this. I wonder if this is also related to:

https://issues.apache.org/jira/browse/NIFI-2015

Seems like there is some case where the UUID is ending up as null.

-Bryan


On Wed, Nov 9, 2016 at 11:57 AM, Conrad Crampton <conrad.crampton@secdata.com<mailto:conrad.crampton@secdata.com>>
wrote:
Hi,
I saw this error after I upgraded to 1.0.0 but thought it was maybe due to the issues I had
with that upgrade (entirely my fault it turns out!), but I have seen it a number of times
since so I turned debugging on to get a better stacktrace. Relevant log section as below.
Nothing out of the ordinary, and I never saw this in v0.6.1 or below.
I would have raised a Jira issue, but after logging in to Jira it only let me create a service
desk request (which didn’t sound right).
Regards
Conrad

2016-11-09 16:43:46,413 DEBUG [Timer-Driven Process Thread-5] o.a.n.processors.standard.MergeContent
MergeContent[id=12c0bec7-68b7-3b60-a020-afcc7b4599e7] has chosen to yield its resources; will
not be scheduled to run again for 1000 milliseconds
2016-11-09 16:43:46,414 DEBUG [Timer-Driven Process Thread-5] o.a.n.processors.standard.MergeContent
MergeContent[id=8db3bb68-0354-3116-96c5-dc80854ef116] Binned 42 FlowFiles
2016-11-09 16:43:46,418 INFO [Timer-Driven Process Thread-5] o.a.n.processors.standard.MergeContent
MergeContent[id=8db3bb68-0354-3116-96c5-dc80854ef116] Merged [StandardFlowFileRecord[uuid=5e846136-0a7a-46fb-be96-8200d5cdd33d,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=567158, length=2337],offset=0,name=17453303363322987,size=2337], StandardFlowFileRecord[uuid=a5f4bd55-82e3-40cb-9fa9-86b9e6816f67,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=573643, length=2279],offset=0,name=17453303351196175,size=2279], StandardFlowFileRecord[uuid=c1ca745b-660a-49cd-82e5-fa8b9a2f4165,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=583957, length=2223],offset=0,name=17453303531879367,size=2223], StandardFlowFileRecord[uuid=<null>,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=595617, length=2356],offset=0,name=<null>,size=2356], StandardFlowFileRecord[uuid=<null>,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=705637, length=2317],offset=0,name=<null>,size=2317], StandardFlowFileRecord[uuid=<null>,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=725376, length=2333],offset=0,name=<null>,size=2333], StandardFlowFileRecord[uuid=<null>,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1475059643340-275849, container=default, section=393],
offset=728703, length=2377],offset=0,name=<null>,size=2377]] into StandardFlowFileRecord[uuid=1ef3e5a0-f8db-49eb-935d-ed3c991fd631,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1478709819819-416, container=default, section=416],
offset=982498, length=4576],offset=0,name=3649103647775837,size=4576]
2016-11-09 16:43:46,418 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.standard.MergeContent
MergeContent[id=8db3bb68-0354-3116-96c5-dc80854ef116] MergeContent[id=8db3bb68-0354-3116-96c5-dc80854ef116]
failed to process session due to java.lang.NullPointerException: java.lang.NullPointerException
2016-11-09 16:43:46,422 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.standard.MergeContent
java.lang.NullPointerException: null
        at org.apache.nifi.stream.io<http://org.apache.nifi.stream.io>.DataOutputStream.writeUTF(DataOutputStream.java:300)
~[nifi-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.stream.io<http://org.apache.nifi.stream.io>.DataOutputStream.writeUTF(DataOutputStream.java:281)
~[nifi-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.provenance.StandardRecordWriter.writeUUID(StandardRecordWriter.java:257)
~[na:na]
        at org.apache.nifi.provenance.StandardRecordWriter.writeUUIDs(StandardRecordWriter.java:266)
~[na:na]
        at org.apache.nifi.provenance.StandardRecordWriter.writeRecord(StandardRecordWriter.java:232)
~[na:na]
        at org.apache.nifi.provenance.PersistentProvenanceRepository.persistRecord(PersistentProvenanceRepository.java:766)
~[na:na]
        at org.apache.nifi.provenance.PersistentProvenanceRepository.registerEvents(PersistentProvenanceRepository.java:432)
~[na:na]
        at org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:713)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:311)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:299)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:256) ~[nifi-processor-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:190) ~[nifi-processor-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_51]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_51]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_51]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_51]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
2016-11-09 16:43:46,422 WARN [Timer-Driven Process Thread-5] o.a.n.processors.standard.MergeContent
MergeContent[id=8db3bb68-0354-3116-96c5-dc80854ef116] Processor Administratively Yielded for
1 sec due to processing failure
2016-11-09 16:43:46,422 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
Administratively Yielding MergeContent[id=8db3bb68-0354-3116-96c5-dc80854ef116] due to uncaught
Exception: java.lang.NullPointerException
2016-11-09 16:43:46,423 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.NullPointerException: null
        at org.apache.nifi.stream.io<http://org.apache.nifi.stream.io>.DataOutputStream.writeUTF(DataOutputStream.java:300)
~[nifi-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.stream.io<http://org.apache.nifi.stream.io>.DataOutputStream.writeUTF(DataOutputStream.java:281)
~[nifi-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.provenance.StandardRecordWriter.writeUUID(StandardRecordWriter.java:257)
~[na:na]
        at org.apache.nifi.provenance.StandardRecordWriter.writeUUIDs(StandardRecordWriter.java:266)
~[na:na]
        at org.apache.nifi.provenance.StandardRecordWriter.writeRecord(StandardRecordWriter.java:232)
~[na:na]
        at org.apache.nifi.provenance.PersistentProvenanceRepository.persistRecord(PersistentProvenanceRepository.java:766)
~[na:na]
        at org.apache.nifi.provenance.PersistentProvenanceRepository.registerEvents(PersistentProvenanceRepository.java:432)
~[na:na]
        at org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:713)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:311)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:299)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:256) ~[nifi-processor-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:190) ~[nifi-processor-utils-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_51]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_51]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_51]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_51]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]

SecureData, combating cyber threats

________________________________

The information contained in this message or any of its attachments may be privileged and
confidential and intended for the exclusive use of the intended recipient. If you are not
the intended recipient any disclosure, reproduction, distribution or other dissemination or
use of this communications is strictly prohibited. The views expressed in this email are those
of the individual and not necessarily of SecureData Europe Ltd. Any prices quoted are only
valid if followed up by a formal written quote.

SecureData Europe Limited. Registered in England & Wales 04365896. Registered Address:
SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, ME16 9NT



***This email originated outside SecureData***

Click here<https://www.mailcontrol.com/sr/g95jBBqyjhnGX2PQPOmvUmQbZlJUA6MY1yb7Iph5QfZYYR83anKczOVB9gK6aeT3tbUbeG4ZHFX1VsBVsOva5A==>
to report this email as spam.
Mime
View raw message