nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aldrin Piri <aldrinp...@gmail.com>
Subject Re: Dynamic Properties for Processor
Date Thu, 24 Sep 2015 13:57:46 GMT
You certainly are on the right path and have the right ideas, but there are
some details to work out.  The biggest item here is that GetTwitter does
not support dynamic properties for the credentials, largely because it does
not take FlowFiles in as input to the processor.  GetTwitter falls into the
class of "source" processors. Ultimately, I'd imagine this was due to how
the Twitter client library interacts with the service and keeping that
connection managed for continuously streaming data.  As a result, the
GetTwitter processor would have to be configured with the appropriate
credentials by hand as it is not currently possible to configure and
connect dynamically.

Some other points to clarify issues you are seeing:

Given EvaluateJsonPath's contract, it is interpreting your Directory
dynamic property you specified in its configuration as a JsonPath
expression, which in this case is invalid given the value you supplied.
You have the right idea to set that Directory property for use in your
PutFile, but that particular property would be better suited with an
UpdateAttribute that makes use of a value that was extracted by
EvaluateJsonPath.  For instance, your directory property in the
PutAttribute could use the "User Id" attribute via NiFi's Expression
Language [1].  Your directory property for your PutFile would then be
/Feeds/${User Id} or something similar depending on where on the filesystem
you wish for it to be rooted.

Let us know if this generates additional questions.

--aldrin

[1]
http://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html



On Thu, Sep 24, 2015 at 7:52 AM, wizz <robertus@johansyah.com> wrote:

> Hi Aldrin and Joe,
>
> Thanks for the clarification. I think the best is to show by picture. I
> made
> NiFi flow graph, please find it in this screenshot (1.png).
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/n2955/1.png>
>
> It is not working one but it will do to explain what I am trying to
> achieve.
>
> The goal that I would like is to get JSON values from JSON, it contains the
> Twitter credentials, get the latest status updates of an user and save them
> to file based on the user ID.
>
> Here is the JSON format:
> {
>     "user_id": 123,
>     "key": "ABC",
>     "secret": "XYZ",
>     "token": "AAA",
>     "tokensecret: "BBB",
>     "twitter_userid": "1234"
> }
>
> I attached what is the settings for the EvaluateJsonPath, (2.png)
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/n2955/2.png>
>
> And also, the error that I get which the directory property is missing,
> which I am hoping that the property can be set from the EvaluateJsonPath.
> (3.png)
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/n2955/3.png>
>
> And the last processor is the PutFile (4.png)
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/n2955/4.png>
>
> I am sure I did something wrong here in terms of the NiFi concepts and
> goals. This is one of the example, I would like to use NiFi for. By having
> dynamic values from external data (such as JSON), dictate what the NiFi
> should be doing (giving the key, token for Twitter processor for a example)
> and where to put the file (based on the User ID from JSON)
>
> It would be amazing and appreciated if you guys could direct me and suggest
> me.
>
> Thanks!!
>
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Dynamic-Properties-for-Processor-tp2895p2955.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message