nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@apache.org>
Subject Re: Someone recommend a good Avro studdy guide for newbie?
Date Wed, 22 Nov 2017 21:28:47 GMT
Eric,

If you're looking for examples on implementing a scripted record
reader/writer, you can see the unit test examples [1] or Drew Lim's
blog [2].  However I suspect you are looking to leverage an existing
RecordReader/Writer from a scripted processor such as ExecuteScript or
InvokeScriptedProcessor, to prototype a "record-aware" processor.  A
general approach is implemented in AbstractRecordProcessor[3], I don't
believe the NiFi Scripting NAR has access to that class, but you can
certainly copy-and-paste the relevant bits of code.  I don't think you
can (easily) get access to an existing RecordReader/Writer controller
service, but Mark Payne outlined the alternative in your mailing list
question about getting a DBCPService property. The recommended
approach is to use InvokeScriptedProcessor and specify your own
RecordReader/Writer properties which will be presented to the user in
the InvokeScriptedProcessor dialog (once the script has been applied).
Then you can get the RecordReader/Writer objects (see [3] lines
100-101) and use them in a similar way as [3].

This (and the answer(s) to your questions in the other mailing list
post) would make excellent blog posts/articles, I hope to do some
relatively soon, will keep the list posted :)

Regards,
Matt

[1] https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/resources/groovy
[2] https://community.hortonworks.com/articles/115311/convert-csv-to-json-avro-xml-using-convertrecord-p.html
[3] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractRecordProcessor.java#L94


On Wed, Nov 22, 2017 at 3:00 PM, Eric Chaves <eric@uolet.com> wrote:
> Hi folks,
>
> Due to my lack of knowledge with Avro in my scripted processors I'm required
> to convert them to JSON in order to be able to manipulate them.
>
> I would like to avoid this and write record based processors but I have no
> clue about how should I do to actually read/write/modify record based
> content (including modifying underlying schemas).
>
> Do you have any reading material to recommend?
>
> Thanks in advance,
>
> Eric

Mime
View raw message