nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dnyaneshwar Pawar <dnyaneshwar_pa...@persistent.com>
Subject RE: GetMongo failure routing issue.
Date Wed, 30 Jan 2019 13:27:31 GMT
Hi Mike,

   Thanks for the support. Just to report new test cases, we tried below:


  1.  Invalid URL: If GetMongo processor is configured with invalid “MongoURL” in controller
service, while fetching data, it gives ERROR in Bulletin and log file (Ex. Connection Refused).
But the flow stuck, instead of routing to the failure relation and the request flow file remains
in QUEUE.



  1.  Invalid Database/Collection names:  If GetMongo is configured with invalid DatabaseName/CollectionName,
it doesn't log any ERROR as bulletin or in log file and also doesn't route to failure relation.
It should route to failure with some Error message.



  1.  Invalid Query [Existing issue we discussed]:  We tried the below solution you have provided
and it is working fine. However, do we need to raise the JIRA, to get it fixed by the way
of pulling it off from the attribute?

Regards,
Dnyaneshwar Pawar

From: Mike Thomsen <mikerthomsen@gmail.com>
Sent: Wednesday, January 30, 2019 2:14 AM
To: users@nifi.apache.org
Subject: Re: GetMongo failure routing issue.

Ok, figured it out. I missed the fact that you were pulling it off an attribute. I can reproduce
the bug now. For now, a work around would be to put it in the body of the flowfile. That doesn't
seem to have any problems with handling malformed queries.

Thanks,

Mike

On Tue, Jan 29, 2019 at 3:40 PM Mike Thomsen <mikerthomsen@gmail.com<mailto:mikerthomsen@gmail.com>>
wrote:
I just passed a query through the flowfile body with the value {"input":} and it routed to
failure without incident. Anything else about your environment you can share?

On Mon, Jan 28, 2019 at 1:55 AM Dnyaneshwar Pawar <dnyaneshwar_pawar@persistent.com<mailto:dnyaneshwar_pawar@persistent.com>>
wrote:
Hi Mike,

We are sending input through the post body from our application. Below is the complete request,
the yellow highlighted query value would go as input to GetMongo:

{
  "QUERY" : "{\"input1\":}"
}

Below is the error logs:

2019-01-28 12:02:41,559 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.mongodb.GetMongo
GetMongo[id=a14b5120-8924-3892-0de1-7279148dba2b] GetMongo[id=a14b5120-8924-3892-0de1-7279148dba2b]
failed to process session due to org.bson.json.JsonParseException: JSON reader was expecting
a value but found '}'.; Processor Administratively Yielded for 1 sec: org.bson.json.JsonParseException:
JSON reader was expecting a value but found '}'.
org.bson.json.JsonParseException: JSON reader was expecting a value but found '}'.
        at org.bson.json.JsonReader.readBsonType(JsonReader.java:228)
        at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:139)
        at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
        at org.bson.Document.parse(Document.java:105)
        at org.bson.Document.parse(Document.java:90)
        at org.apache.nifi.processors.mongodb.GetMongo.getQuery(GetMongo.java:326)
        at org.apache.nifi.processors.mongodb.GetMongo.onTrigger(GetMongo.java:225)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)


We have also enclosed screen shots for NiFi flow, GetMongo configuration and MongoDB console
output with same query. PFA.

Regards,
Dnyaneshwar Pawar

From: Mike Thomsen <mikerthomsen@gmail.com<mailto:mikerthomsen@gmail.com>>
Sent: Saturday, January 26, 2019 5:48 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: GetMongo failure routing issue.

I just tried to replicate with GenerateFlowfile and GetMongo, and couldn't reproduce it. I
used these examples:

{ "invalid": true }
{ invalid: true }
[ invalid: true ]

The last two are clearly not valid JSON, but Mongo's client API can apparently parse the middle
one.

The first two routed to success, the last one routed to failure both on 1.8.0 and 1.9.0-SNAPSHOT.

Can you share your input?

Thanks,

Mike

On Fri, Jan 25, 2019 at 10:53 AM Mike Thomsen <mikerthomsen@gmail.com<mailto:mikerthomsen@gmail.com>>
wrote:
Ok, so it's a current issue. I'll look into it.

On Fri, Jan 25, 2019 at 12:16 AM Dnyaneshwar Pawar <dnyaneshwar_pawar@persistent.com<mailto:dnyaneshwar_pawar@persistent.com>>
wrote:
Mike,

   We are using MongoDB 3.4.7 and Apache NiFi 1.8.0

Regards,
Dnyaneshwar Pawar


From: Mike Thomsen <mikerthomsen@gmail.com<mailto:mikerthomsen@gmail.com>>
Sent: Thursday, January 24, 2019 8:44 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: GetMongo failure routing issue.

What version are you using?

On Thu, Jan 24, 2019 at 8:23 AM Dnyaneshwar Pawar <dnyaneshwar_pawar@persistent.com<mailto:dnyaneshwar_pawar@persistent.com>>
wrote:
Hi All,
We are working with MongoDB integration with NiFi (Version 1.8). And we observed one issue
related to failure case. Whenever, GetMongo  processes the incorrect query (for example. Invalid
json), the bulletin gets updated with appropriate error message, however, the flow stuck,
instead of routing to the failure relation.

Any pointer?

Regards,
Dnyaneshwar Pawar

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent
Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed.
If you are not the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in error, please notify
the sender and delete all copies of this message. Persistent Systems Ltd. does not accept
any liability for virus infected mails.
Mime
View raw message