nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Taft <a...@adamtaft.com>
Subject Re: Start InvokeHTTP using Cron and run the InvokeHTTP multiple times for paginated results
Date Fri, 11 Oct 2019 00:52:30 GMT
Devaraj,

I've created a mockup of the flow you might use, with some notes to help
you.

https://tinyurl.com/y6gp5r4z

Notice how you start the dataflow using GenerateFlowFile, call InvokeHTTP
with the initial URL, then process the results.  When you extract the
results, create a new attribute for the "nextPage" and update the URL,
looping back into InvokeHTTP.  Something like this should get you close to
what you're looking for.

Hope this helps,
Adam


On Thu, Oct 10, 2019 at 2:45 PM dkommineni <dkommineni@gmail.com> wrote:

> Hi All,
>
> I am looking for a solution to below problem.
>
> I want to have cron job (starts at 2 AM daily) which pulls data from
> certain
> API using GET request.
> The results will be sent to downstream component and if there are more
> results to pull based on nextpage key in the response, I need to call the
> API with additional query parameter mentioning the nextpage key and repeat
> the process until nextpage key is empty or null in the response.
>
> Please provide some pointers to this problem approach.
>
> regards
> Devaraj
>
>
>
> --
> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/
>

Mime
View raw message