nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Madhire, Naveen" <Naveen.Madh...@capitalone.com>
Subject Re: Processor to use
Date Thu, 05 Nov 2015 22:13:19 GMT
The data format is JSON coming from Kafka. Is there any processor for actually matching the
contents of the flow files against a lookup file ?


From: Bryan Bende <bbende@gmail.com<mailto:bbende@gmail.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Date: Thursday, November 5, 2015 at 4:07 PM
To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>>
Subject: Re: Processor to use

Hello,

Can you share the data format coming from Kafka? json, avro, something else?

The reason I ask is because there are many different processors that can modify FlowFiles
in different ways, but a lot depends on the type of data involved.

For general text-based operations we have...
- ReplaceText to modify the content of the FlowFile
- ExtractText to extract parts of the content into FlowFile attributes
- UpdateAttribute to add/update a FlowFile attribute, can use expression language here to
do dynamic stuff

There are processors that interact with json, xml, and avro as well.

If you have a need that is outside all of the existing processors, it is also possible to
develop your own custom processor. Let us know and we can point you in the right direction.

-Bryan


On Thu, Nov 5, 2015 at 4:55 PM, Madhire, Naveen <Naveen.Madhire@capitalone.com<mailto:Naveen.Madhire@capitalone.com>>
wrote:
Hi All,

Just need few suggestions on which Processor to use. I have the below need and I need to figure
out which processor to use in the 2nd point as I don’t see any processor to write customize
logic.


 1.  Read the data from Kafka Topic ( I will use the getKafka processor)
 2.  Apply business logic to the data fetched from Kafka like few calculations (Here I would
need which processor to use to modify the flow files on the flow)
 3.  Write the data into HDFS. (putHDFS processor)

Is there any processor which I can use to make the changes to the flow files?

Thanks.

________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates and may only be used solely in performance of work or services for Capital
One. The information transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended recipient, you
are hereby notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is strictly prohibited.
If you have received this communication in error, please contact the sender and delete the
material from your computer.

________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One
and/or its affiliates and may only be used solely in performance of work or services for Capital
One. The information transmitted herewith is intended only for use by the individual or entity
to which it is addressed. If the reader of this message is not the intended recipient, you
are hereby notified that any review, retransmission, dissemination, distribution, copying
or other use of, or taking of any action in reliance upon this information is strictly prohibited.
If you have received this communication in error, please contact the sender and delete the
material from your computer.

Mime
View raw message