nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: Credentials Oauth
Date Tue, 14 Aug 2018 01:40:13 GMT
I also realize the sleep may seem extraneous with the flowfile penalization that NiFi offers,
but I am not sure if applying that penalty is identical across relationships, so I didn’t
want to penalize the “not-found” relationship that went to the Get Token processor. You
may be interested in the open feature request for a PenalizeFlowFile processor I found here
[1].

[1] https://issues.apache.org/jira/browse/NIFI-4805 <https://issues.apache.org/jira/browse/NIFI-4805>

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 13, 2018, at 6:36 PM, Andy LoPresto <alopresto@apache.org> wrote:
> 
> Hi Travis,
> 
> I have not executed this flow myself, so this is hypothetical — could you use the DistributedMapCache
services to persist the token when you acquire it and retrieve the token from there before
running the second InvokeHTTP processor (the “POST" processor)? You would remove the first
InvokeHTTP (the “token” processor) from the flow directly and have it alongside the flow
with a trigger every 55 minutes or whatever you like, which would persist it to the cache.
In your “main” flow, you can use a FetchDistributedMapCache processor to pull the existing
token and use that?
> 
> I’ve attached a screenshot, but again, I have not run this flow to verify.
> 
> In this case, the Get OAuth Token Every 55 min does exactly what it says. Each time it
runs, it puts that token value in the DMC (you probably need an ExtractText or ReplaceText
processor in between; I can’t remember if the OAuth Token comes back in the response content
or as a header). On the right side, you have the ListFile -> FetchFile, and then it goes
to Fetch Token From DMC. If the token is there, it routes the flowfile to success relationship
and POSTs the file. If the token is not there, it sends the flowfile to two processors: (realizing
now this isn’t quite complete) the Get Token processor and an ExecuteScript processor set
up with a simple Groovy script to sleep for 5 seconds. You would actually need to change how
the Get Token processor runs for this to work; it should be event driven, and there should
be a GenerateFlowFile processor leading to it that is timer driven every 55 minutes. That
way, either the normal scheduling flowfile arriving or the “no token present” flowfile
would trigger the token retrieval. Anyway, once the token is back in the DMC, the sleep processor
should finish and the Fetch Token will try to run again, this time getting the token and triggering
the POST.
> 
> Hope this works for you. If you have to many any adjustments to this, please reply here
to document what works for you.
> 
> <Screen Shot 2018-08-13 at 6.30.23 PM.png>
> 
> 
> 
> Andy LoPresto
> alopresto@apache.org <mailto:alopresto@apache.org>
> alopresto.apache@gmail.com <mailto:alopresto.apache@gmail.com>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Aug 13, 2018, at 5:47 PM, Travis Vaske <travis.vaske@gmail.com <mailto:travis.vaske@gmail.com>>
wrote:
>> 
>> I have a flow that looks like this
>> 
>> list file -> get file -> invokehttp (gets oauth token) -> invokehttp (post
file with token)
>> 
>> This works fine, but my token is good for a certain amount of time (maybe 1 hour),
is there a setting where 1 processor runs every 30 minutes (to get the token) but when a get
file triggers it, it will pass the token that i already have without invoking every time?
>> 
>> Thanks!
> 


Mime
View raw message