nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Thomsen <mikerthom...@gmail.com>
Subject Re: GetMongo failure routing issue.
Date Wed, 30 Jan 2019 14:25:26 GMT
Thanks. I'll take a look at those issues.

On Wed, Jan 30, 2019 at 8:33 AM Dnyaneshwar Pawar <
dnyaneshwar_pawar@persistent.com> wrote:

> Hi Mike,
>
>
>
>    Thanks for the support. Just to report new test cases, we tried below:
>
>
>
>    1. Invalid URL: If GetMongo processor is configured with invalid
>    “MongoURL” in controller service, while fetching data, it gives ERROR in
>    Bulletin and log file (Ex. Connection Refused). But the flow stuck, instead
>    of routing to the failure relation and the request flow file remains in
>    QUEUE.
>
>
>
>    1. Invalid Database/Collection names:  If GetMongo is configured with
>    invalid DatabaseName/CollectionName, it doesn't log any ERROR as bulletin
>    or in log file and also doesn't route to failure relation. It should route
>    to failure with some Error message.
>
>
>
>    1. Invalid Query [Existing issue we discussed]:  We tried the below
>    solution you have provided and it is working fine. However, do we need to
>    raise the JIRA, to get it fixed by the way of pulling it off from the
>    attribute?
>
>
>
> Regards,
>
> Dnyaneshwar Pawar
>
>
>
> *From:* Mike Thomsen <mikerthomsen@gmail.com>
> *Sent:* Wednesday, January 30, 2019 2:14 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: GetMongo failure routing issue.
>
>
>
> Ok, figured it out. I missed the fact that you were pulling it off an
> attribute. I can reproduce the bug now. For now, a work around would be to
> put it in the body of the flowfile. That doesn't seem to have any problems
> with handling malformed queries.
>
>
>
> Thanks,
>
>
>
> Mike
>
>
>
> On Tue, Jan 29, 2019 at 3:40 PM Mike Thomsen <mikerthomsen@gmail.com>
> wrote:
>
> I just passed a query through the flowfile body with the value {"input":}
> and it routed to failure without incident. Anything else about your
> environment you can share?
>
>
>
> On Mon, Jan 28, 2019 at 1:55 AM Dnyaneshwar Pawar <
> dnyaneshwar_pawar@persistent.com> wrote:
>
> Hi Mike,
>
>
>
> We are sending input through the post body from our application. Below is
> the complete request, the yellow highlighted query value would go as input
> to GetMongo:
>
>
>
> {
>
>   "QUERY" : "{\"input1\":}"
>
> }
>
>
>
> Below is the error logs:
>
>
>
> 2019-01-28 12:02:41,559 ERROR [Timer-Driven Process Thread-5]
> o.a.nifi.processors.mongodb.GetMongo
> GetMongo[id=a14b5120-8924-3892-0de1-7279148dba2b]
> GetMongo[id=a14b5120-8924-3892-0de1-7279148dba2b] failed to process session
> due to org.bson.json.JsonParseException: JSON reader was expecting a value
> but found '}'.; Processor Administratively Yielded for 1 sec:
> org.bson.json.JsonParseException: JSON reader was expecting a value but
> found '}'.
>
> org.bson.json.JsonParseException: JSON reader was expecting a value but
> found '}'.
>
>         at org.bson.json.JsonReader.readBsonType(JsonReader.java:228)
>
>         at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:139)
>
>         at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:45)
>
>         at org.bson.Document.parse(Document.java:105)
>
>         at org.bson.Document.parse(Document.java:90)
>
>         at
> org.apache.nifi.processors.mongodb.GetMongo.getQuery(GetMongo.java:326)
>
>         at
> org.apache.nifi.processors.mongodb.GetMongo.onTrigger(GetMongo.java:225)
>
>         at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>
>         at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
>
>         at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
>
>         at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
>
>         at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
>       at java.lang.Thread.run(Thread.java:745)
>
>
>
>
>
> We have also enclosed screen shots for NiFi flow, GetMongo configuration
> and MongoDB console output with same query. PFA.
>
>
>
> Regards,
>
> Dnyaneshwar Pawar
>
>
>
> *From:* Mike Thomsen <mikerthomsen@gmail.com>
> *Sent:* Saturday, January 26, 2019 5:48 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: GetMongo failure routing issue.
>
>
>
> I just tried to replicate with GenerateFlowfile and GetMongo, and couldn't
> reproduce it. I used these examples:
>
>
>
> { "invalid": true }
>
> { invalid: true }
>
> [ invalid: true ]
>
>
>
> The last two are clearly not valid JSON, but Mongo's client API can
> apparently parse the middle one.
>
>
>
> The first two routed to success, the last one routed to failure both on
> 1.8.0 and 1.9.0-SNAPSHOT.
>
>
>
> Can you share your input?
>
>
>
> Thanks,
>
>
>
> Mike
>
>
>
> On Fri, Jan 25, 2019 at 10:53 AM Mike Thomsen <mikerthomsen@gmail.com>
> wrote:
>
> Ok, so it's a current issue. I'll look into it.
>
>
>
> On Fri, Jan 25, 2019 at 12:16 AM Dnyaneshwar Pawar <
> dnyaneshwar_pawar@persistent.com> wrote:
>
> Mike,
>
>
>
>    We are using MongoDB 3.4.7 and Apache NiFi 1.8.0
>
>
>
> Regards,
>
> Dnyaneshwar Pawar
>
>
>
>
>
> *From:* Mike Thomsen <mikerthomsen@gmail.com>
> *Sent:* Thursday, January 24, 2019 8:44 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: GetMongo failure routing issue.
>
>
>
> What version are you using?
>
>
>
> On Thu, Jan 24, 2019 at 8:23 AM Dnyaneshwar Pawar <
> dnyaneshwar_pawar@persistent.com> wrote:
>
> Hi All,
>
> We are working with MongoDB integration with NiFi (Version 1.8). And we
> observed one issue related to failure case. Whenever, GetMongo  processes
> the incorrect query (for example. Invalid json), the bulletin gets updated
> with appropriate error message, however, the flow stuck, instead of routing
> to the failure relation.
>
>
>
> Any pointer?
>
>
>
> Regards,
>
> Dnyaneshwar Pawar
>
>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>
>

Mime
View raw message