nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Ryan <brett.r...@gmail.com>
Subject Re: Enrichment plugin for adding attributes from SQL
Date Thu, 04 Jan 2018 21:07:06 GMT
Thanks Andy, how would update attribute be able to get the value from sql?

Consider a flow where a piece of information needs to be obtained from a DB but i do not want
the contents of the current FF to be altered, using ExecuteSQL anywhere prior would not be
possible due to replacing the FF contents.

What I had was two seperate flows, one that updates an oauth key in a db keeping it fresh,
the main flow would then read the db just before an invokehttp.

I was originally using a distributed map cache but had concerns that it might not be secure
and was also advised the cache server has been known to go down.

> On 5 Jan 2018, at 04:01, Joey Frazee <joey.frazee@icloud.com> wrote:
> 
> Andy, Brett,
> 
> Taking a quick glance at the code it looks like it's enriching attributes from a database
according to a query.
> 
> If that's correct, there's a LookupAttribute processor that delegates lookups to a "LookupService"
and adds attributes without altering content. There are a variety of these LookupServices
included. I think what you implemented would make sense as a LookupService and then you could
just configure the processor to use that. It could also be used with LookupRecord then too
so there'd be a double payoff.
> 
> -joey
> 
>> On Jan 4, 2018, 8:44 AM -0800, Andy LoPresto <alopresto@apache.org>, wrote:
>> Hi Brett,
>> 
>> It’s great that you found it easy to write a new processor for Apache NiFi. It
is probably an indicator that we need to improve education/evangelism/documentation, however,
that you did not find UpdateAttribute [1], which should do exactly what you were looking for.
>> 
>> [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-update-attribute-nar/1.4.0/org.apache.nifi.processors.attributes.UpdateAttribute/index.html
>> 
>> Andy LoPresto
>> alopresto@apache.org
>> alopresto.apache@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>>> On Jan 4, 2018, at 7:03 AM, Brett Ryan <brett.ryan@gmail.com> wrote:
>>> 
>>> Hi all, having used NiFi for a couple days I wanted to add some attributes to
a FlowFile while not altering the contents of that FlowFile.
>>> 
>>> I had suggestions to use a script processor but that just sounded like a hack
which could become a nuisance to replicate.
>>> 
>>> Anyway, I figured I'd write a processor to do this, anyone interested you can
find it here.
>>> 
>>> https://github.com/brettryan/nifi-drunken-bundle
>>> 
>>> Feel free to do with it what you please.
>>> 
>>> I've published to maven central but it will take a day to appear in the search.
>>> 
>>> <dependency>
>>>   <groupId>com.drunkendev</groupId>
>>>   <artifactId>nifi-drunken-nar</artifactId>
>>>   <version>1.0.0</version>
>>>   <type>nar</type>
>>> </dependency>
>>> <dependency>
>>>   <groupId>com.drunkendev</groupId>
>>>   <artifactId>nifi-drunken-processors</artifactId>
>>>   <version>1.0.0</version>
>>> </dependency>
>>> <dependency>
>>>   <groupId>com.drunkendev</groupId>
>>>   <artifactId>nifi-drunken-bundle</artifactId>
>>>   <version>1.0.0</version>
>>>   <type>pom</type>
>>> </dependency>
>>> 
>> 

Mime
View raw message