nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Geercken" <uwe.geerc...@web.de>
Subject Aw: Re: Re: Nifi 1.4: problem with QueryRecord Precessor
Date Wed, 04 Oct 2017 15:46:08 GMT
<html><head></head><body><div style="font-family: Verdana;font-size:
12.0px;"><div>&nbsp;
<div>
<div>thanks Matt.</div>

<div>&nbsp;</div>

<div>Uwe</div>

<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px
solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Gesendet:</b>&nbsp;Mittwoch, 04.
Oktober 2017 um 17:32 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Matt Burgess&quot; &lt;mattyb149@apache.org&gt;<br/>
<b>An:</b>&nbsp;users@nifi.apache.org<br/>
<b>Betreff:</b>&nbsp;Re: Re: Nifi 1.4: problem with QueryRecord Precessor</div>

<div name="quoted-content">All,<br/>
<br/>
The known issue Mark is referring to is NIFI-4349 [1], however it is<br/>
not causing the problem; rather it is hiding the problem. If an error<br/>
occurs (due to misonfiguration, schema errors, etc.), certain cleanup<br/>
activities aren&#39;t being performed so the processor ends up with the<br/>
error about processing flow files, InputStreams, etc. instead.<br/>
Regarding it working in 1.3.0 and not in 1.4.0, I think some of the<br/>
logic changes in NIFI-3921 [2] may have changed the handling of<br/>
inherited schemas and now generates an exception where it may have<br/>
been ignored before.<br/>
<br/>
Regards,<br/>
Matt<br/>
<br/>
[1] <a href="https://issues.apache.org/jira/browse/NIFI-4349" target="_blank">https://issues.apache.org/jira/browse/NIFI-4349</a><br/>
[2] <a href="https://issues.apache.org/jira/browse/NIFI-3921" target="_blank">https://issues.apache.org/jira/browse/NIFI-3921</a><br/>
<br/>
On Wed, Oct 4, 2017 at 11:22 AM, Uwe Geercken &lt;uwe.geercken@web.de&gt; wrote:<br/>
&gt;<br/>
&gt; Mark,<br/>
&gt;<br/>
&gt; I stopped 1.4 and started 1.3 and created the same flow. And in 1.3 it works<br/>
&gt; without a problem.<br/>
&gt;<br/>
&gt; And I looked into the 1.4 log (I should have looked before, but only looked<br/>
&gt; at the processor&#39;s tooltip). You are right, it&#39;s a problem with the
schema.<br/>
&gt; Here is the real error:<br/>
&gt;<br/>
&gt; 2017-10-04 16:01:16,640 ERROR [Timer-Driven Process Thread-10]<br/>
&gt; o.a.nifi.processors.standard.QueryRecord<br/>
&gt; QueryRecord[id=e798adc7-015e-1000-a43d-68a02b5d425a] Unable to qu&#36;<br/>
&gt; org.apache.nifi.processor.exception.ProcessException: IOException thrown<br/>
&gt; from QueryRecord[id=e798adc7-015e-1000-a43d-68a02b5d425a]:<br/>
&gt; java.io.IOException: org.apache.nifi.&#36;<br/>
&gt; at<br/>
&gt; org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2589)<br/>
&gt; at<br/>
&gt; org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:284)<br/>
&gt; at<br/>
&gt; org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)<br/>
&gt; at<br/>
&gt; org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)<br/>
&gt; at<br/>
&gt; org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)<br/>
&gt; at<br/>
&gt; org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)<br/>
&gt; at<br/>
&gt; org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent&#36;1.run(TimerDrivenSchedulingAgent.java:128)<br/>
&gt; at<br/>
&gt; java.util.concurrent.Executors&#36;RunnableAdapter.call(Executors.java:511)<br/>
&gt; at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)<br/>
&gt; at<br/>
&gt; java.util.concurrent.ScheduledThreadPoolExecutor&#36;ScheduledFutureTask.access&#36;301(ScheduledThreadPoolExecutor.java:180)<br/>
&gt; at<br/>
&gt; java.util.concurrent.ScheduledThreadPoolExecutor&#36;ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)<br/>
&gt; at<br/>
&gt; java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)<br/>
&gt; at<br/>
&gt; java.util.concurrent.ThreadPoolExecutor&#36;Worker.run(ThreadPoolExecutor.java:624)<br/>
&gt; at java.lang.Thread.run(Thread.java:748)<br/>
&gt; Caused by: java.io.IOException:<br/>
&gt; org.apache.nifi.schema.access.SchemaNotFoundException: Cannot write Schema<br/>
&gt; Name As Attribute because the Schema Name is not known<br/>
&gt; at<br/>
&gt; org.apache.nifi.processors.standard.QueryRecord&#36;1.process(QueryRecord.java:302)<br/>
&gt; at<br/>
&gt; org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2570)<br/>
&gt; ... 13 common frames omitted<br/>
&gt; Caused by: org.apache.nifi.schema.access.SchemaNotFoundException: Cannot<br/>
&gt; write Schema Name As Attribute because the Schema Name is not known<br/>
&gt; at<br/>
&gt; org.apache.nifi.schema.access.SchemaNameAsAttribute.validateSchema(SchemaNameAsAttribute.java:53)<br/>
&gt; at<br/>
&gt; org.apache.nifi.serialization.SchemaRegistryRecordSetWriter.getSchemaAccessWriter(SchemaRegistryRecordSetWriter.java:137)<br/>
&gt; at<br/>
&gt; org.apache.nifi.csv.CSVRecordSetWriter.createWriter(CSVRecordSetWriter.java:72)<br/>
&gt; at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br/>
&gt;<br/>
&gt;<br/>
&gt;<br/>
&gt; I am using an AvroReader using the embedded schema. Maybe this helps.<br/>
&gt;<br/>
&gt; Rgds,<br/>
&gt;<br/>
&gt; Uwe<br/>
&gt;<br/>
&gt;<br/>
&gt; Gesendet: Mittwoch, 04. Oktober 2017 um 16:55 Uhr<br/>
&gt; Von: &quot;Mark Payne&quot; &lt;markap14@hotmail.com&gt;<br/>
&gt; An: &quot;users@nifi.apache.org&quot; &lt;users@nifi.apache.org&gt;<br/>
&gt;<br/>
&gt; Betreff: Re: Nifi 1.4: problem with QueryRecord Precessor<br/>
&gt; Dan, Uwe,<br/>
&gt;<br/>
&gt; This is something that I will be looking into shortly. It is a known issue<br/>
&gt; that exists in both 1.3<br/>
&gt; and 1.4. This occurs whenever there is some other sort of failure, when the<br/>
&gt; processor attempts<br/>
&gt; to route the FlowFile to &#39;failure&#39;. This occurs most often when there
is a<br/>
&gt; problem obtaining the<br/>
&gt; schema for the Record. Can you check your logs and see what other error may<br/>
&gt; be present<br/>
&gt; in the logs?<br/>
&gt;<br/>
&gt; Thanks<br/>
&gt; -Mark<br/>
&gt;<br/>
&gt;<br/>
&gt;<br/>
&gt; On Oct 4, 2017, at 10:39 AM, dan young &lt;danoyoung@gmail.com&gt; wrote:<br/>
&gt;<br/>
&gt;<br/>
&gt; It might be...maybe others can share experience with 1.4...<br/>
&gt;<br/>
&gt;<br/>
&gt; On Wed, Oct 4, 2017, 8:37 AM Uwe Geercken &lt;uwe.geercken@web.de&gt; wrote:<br/>
&gt;&gt;<br/>
&gt;&gt; thanks Dano.<br/>
&gt;&gt;<br/>
&gt;&gt; Any other comments? Is this a bug in 1.4 then?<br/>
&gt;&gt;<br/>
&gt;&gt; Rgds,<br/>
&gt;&gt;<br/>
&gt;&gt; Uwe<br/>
&gt;&gt;<br/>
&gt;&gt; Gesendet: Mittwoch, 04. Oktober 2017 um 16:33 Uhr<br/>
&gt;&gt; Von: &quot;dan young&quot; &lt;danoyoung@gmail.com&gt;<br/>
&gt;&gt; An: nifi &lt;users@nifi.apache.org&gt;<br/>
&gt;&gt; Betreff: Re: Nifi 1.4: problem with QueryRecord Precessor<br/>
&gt;&gt;<br/>
&gt;&gt; We&#39;re seeing the same thing. Works fine in 1.3<br/>
&gt;&gt;<br/>
&gt;&gt; Dano<br/>
&gt;&gt;<br/>
&gt;&gt;<br/>
&gt;&gt; On Wed, Oct 4, 2017, 8:13 AM Uwe Geercken &lt;uwe.geercken@web.de&gt;
wrote:<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; Hello,<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; I have created a flow: GetFile &gt;&gt; QueryRecord &gt;&gt;
Putfile. GetFile reads<br/>
&gt;&gt;&gt; an avro file. QueryRecord has one property/sql and the result is
routed to<br/>
&gt;&gt;&gt; PutFile.<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; When I run the processor, I get following error:<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; failed to process session due to java.lang.IllegalStateException<br/>
&gt;&gt;&gt; ....<br/>
&gt;&gt;&gt; &lt;filename here&gt; already in use or an active callback
or an inputstream<br/>
&gt;&gt;&gt; created by ProcessSession.read(FlowFile) has not been closed.<br/>
&gt;&gt;&gt; ....<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; Can somebody help?<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; Rgds,<br/>
&gt;&gt;&gt;<br/>
&gt;&gt;&gt; Uwe</div>
</div>
</div>
</div></div></body></html>

Mime
View raw message