nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Thomsen <mikerthom...@gmail.com>
Subject Re: Help with NiFi LookupRecord & MergeRecord Processors
Date Tue, 12 Mar 2019 14:22:23 GMT
If you're not a Java developer and/or don't have access to any on your
team, your best work around would be to look at massaging your lookup data
into HBase, Mongo or ElasticSearch and using those. If you are a Java
developer  or have one on your team I can point you to some sample projects
you can hack up to quickly spin up a custom component that is tailored to
your needs.

On Mon, Mar 11, 2019 at 6:57 PM matthewmichala@northwesternmutual.com <
matthewmichala@northwesternmutual.com> wrote:

> FYI, I did get the following responses from the Stack Overflow Nifi Tag
> Site - so until the DB Lookup Service is available, does anyone have any
> other workaround or custom built solutions:
>
> Stack Overflow Reply 1:
>
> When you use LookupRecord, what LookupService are you using? There
> currently is not a released implementation of a LookupService that works
> against a relatonial DB, although there is active work to implement one
>
> Stack Overflow Reply 2:
>
> MergeRecord understands records and can merge them together as proper
> JSON. In both cases though, merging is not joining, merging is writing
> content or records one after another to create larger flow files.
>
>
>
> -----Original Message-----
> From: matthewmichala@northwesternmutual.com <
> matthewmichala@northwesternmutual.com>
> Sent: Monday, March 11, 2019 5:51 PM
> To: dev@nifi.apache.org
> Subject: [EXTERNAL] Help with NiFi LookupRecord & MergeRecord Processors
>
> NiFi Developers,
>
> I'm a senior engineer at Northwestern Mutual in Milwaukee, and was just
> wondering if  anyone can help us with the NiFi LookupRecord, or MergeRecord
> Processors?
>
> We keep getting this error when trying to use any Lookup Service (NiFi or
> 3rd party) to append additional attributes to our initial DB2 Query:
>
>      Failed to lookup coordinates {LEGAL_ENTITY_ID_NUM=13366} in Lookup
> Service
>
> Has anyone had any success merging data from 2 different sql queries based
> on a common Unique ID from both queries?   We've tried LookupRecord which
> uses the Lookup Service, and also tried MergeRecord, and MergeContent -
> nothing seems to work??
>
> Bottom line is - we have an initial main query (ID & Status) - then want
> to append an additional attribute (Name) from a 2nd query (which just
> contains ID & Name) - onto each record of the main query - after joining on
> ID.
>
> Whenever we try using LookupRecord, and pass the ID as a parameter via ?
> marks, we keep getting the above error.  We've gotten nowhere with
> MergeRecord too.  We have learned how to MergeContent, but that is not
> really what we want (is just a workaround).
> Thanks & Regards,
>
> Matt Michala
> Senior Engineer
> Digital Workplace & Corp. Solutions
> Northwestern Mutual (T07-150)
> 414-661-4668
>
>
> Thanks & Regards,
>
> Matt Michala
> Senior Engineer
> Digital Workplace & Corp. Solutions
> Northwestern Mutual (T07-150)
> 414-661-4668
>
>
>
> This e-mail and any attachments may contain confidential information of
> Northwestern Mutual. If you are not the intended recipient of this message,
> be aware that any disclosure, copying, distribution or use of this e-mail
> and any attachments is prohibited. If you have received this e-mail in
> error, please notify Northwestern Mutual immediately by returning it to the
> sender and delete all copies from your system. Please be advised that
> communications with {SECURE MESSAGE} in the subject line have been sent
> using a secure messaging system. Communications that do not have this tag
> may not be secure and could be observed by a third party. Thank you for
> your cooperation.
>
>
>
>
> This e-mail and any attachments may contain confidential information of
> Northwestern Mutual. If you are not the intended recipient of this message,
> be aware that any disclosure, copying, distribution or use of this e-mail
> and any attachments is prohibited. If you have received this e-mail in
> error, please notify Northwestern Mutual immediately by returning it to the
> sender and delete all copies from your system. Please be advised that
> communications with {SECURE MESSAGE} in the subject line have been sent
> using a secure messaging system. Communications that do not have this tag
> may not be secure and could be observed by a third party. Thank you for
> your cooperation.
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message