nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Craig Knell <craig.kn...@gmail.com>
Subject Re: Postgres table as Cached Lookup Service
Date Sun, 11 Aug 2019 23:22:58 GMT
Thanks for the advice I’ll give your groovy script a go.   

Best regards

Craig Knell

Mobile 61 402128615
Skype craigknell

> On 12 Aug 2019, at 00:21, Matt Burgess <mattyb149@apache.org> wrote:
> 
> Sorry just getting back to this, I thought it was in 1.9 but Mike's
> right, it's going into 1.10 [1]. If you're comfortable with a
> scripting language like Groovy or Jython, there are a couple of
> workarounds:
> 
> - ExecuteGroovyScript gives you a "sql" object you can use to do your
> own lookups
> - ScriptedLookupService can be used with LookupRecord, I have an
> example of looking up PostgreSQL sequence values using
> ScriptedLookupService and Groovy [2]
> 
> Regards,
> Matt
> 
> [1] https://issues.apache.org/jira/browse/NIFI-6082
> [2] https://funnifi.blogspot.com/2018/08/database-sequence-lookup-with.html
> 
>> On Sun, Aug 11, 2019 at 11:10 AM Mike Thomsen <mikerthomsen@gmail.com> wrote:
>> 
>> I think it's 1.10. I've seen what he's talking about.
>> 
>>> On Sat, Aug 10, 2019 at 2:30 PM Andrew Grande <aperepel@gmail.com> wrote:
>>> 
>>> Matt, the 1.9.2 docs don't list anything like that. Are you sure? Is it something
coming out in the next, unreleased version? A more than welcome addition :)
>>> 
>>> Silly me was searching for everything jdbc, of course, on the page.
>>> 
>>> Andrew
>>> 
>>>> On Sat, Aug 10, 2019, 11:08 AM Matt Burgess <mattyb149@gmail.com> wrote:
>>>> 
>>>> There is, sorry I’m AFK ATM but there’s a SimpleDatabaseLookup and a
DatabaseRecordLookup (or something similarly named) :)
>>>> 
>>>> Sent from my iPhone
>>>> 
>>>> On Aug 10, 2019, at 1:31 PM, Andrew Grande <aperepel@gmail.com> wrote:
>>>> 
>>>> Maybe this would help? https://github.com/mrcsparker/nifi-sqllookup-services-bundle/blob/master/README.md
>>>> 
>>>> I wish there was a standard bundled jdbc lookup record implementation.
>>>> 
>>>> Andrew
>>>> 
>>>>> On Fri, Aug 9, 2019, 11:56 PM Craig Knell <craig.knell@gmail.com>
wrote:
>>>>> 
>>>>> Hi Folks
>>>>> 
>>>>> Cached Postgres Lookup Service
>>>>> whats the best way to create a postgres lookup service within nifi.
>>>>> 
>>>>> I have an incoming ff with content in JSON.
>>>>> What i want to do is use the field
>>>>> =  "ID" : 12344
>>>>> field to lookup a postgres table and ADD in a new json field
>>>>> =  "NAME" : lookupvalue
>>>>> back into the ff content in json format.
>>>>> 
>>>>> I would ideally like the lookup processor to get/refresh the cached data
daily.
>>>>> 
>>>>> Avro Records
>>>>> It looks like the Rcord processing should work with the Cache,
>>>>> I however get a little lost with using Record processors and AVRO
>>>>> 1. are avro schema's case sensitive?
>>>>> 2. how do i convert from incoming JSON ff with  uppercase fields to an
>>>>> out going ff in lowercase fields ?
>>>>> 3. i performed a validateRecord processor using lowercase avro schema
>>>>> against the incoming ff with uppercase fields and it returned 3 of the
>>>>> 10 fields, in lowercase, with null values, not sure what this means.
>>>>> 
>>>>> Thanks
>>>>> 
>>>>> Craig

Mime
View raw message