nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James McMahon <jsmcmah...@gmail.com>
Subject Re: Reading flowfile in a stream callback
Date Fri, 03 Nov 2017 09:48:36 GMT
Thank you Andy. I'd like to ask just a few quick follow up questions.

1- My flow content may be textual characters, and it can also be binary -
jpgs, pngs, and similar. How can discern binary or character content using
conditional checks to be sure I handle the file properly? How would I alter
this

text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

to read in the data from the stream as binary data in that case?

2- In the case where my data in the flowfile payload is binary, do I have
another version of this....

outputStream.write(bytearray(reversedText.encode('utf-8')))

....that omits the encoding, like so:

outputStream.write(bytearray(some_binary))  ?

Thank you very much in advance. -Jim

On Thu, Nov 2, 2017 at 8:26 PM, Andy LoPresto <alopresto@apache.org> wrote:

> James,
>
> The Python API should be the same as the Java FlowFile.java interface [1].
> Matt Burgess’ blog has a good post about using Jython to do flowfile
> content manipulation. Something like:
>
> flowFile = session.get()
> if (flowFile != None):
>   flowFile = session.write(flowFile,PyStreamCallback())
>   session.transfer(flowFile, REL_SUCCESS)
>
> With PyStreamCallback declared as a class above that block in the script:
>
> import java.io
> from org.apache.commons.io import IOUtils
> from java.nio.charset import StandardCharsets
> from org.apache.nifi.processor.io import StreamCallback
>
> class PyStreamCallback(StreamCallback):
>   def __init__(self):
>         pass
>   def process(self, inputStream, outputStream):
>     text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
>     reversedText = text[::-1]
>
>     outputStream.write(bytearray(reversedText.encode('utf-8')))
>
> In Groovy, you can declare the StreamCallback as an inline closure to make
> this more compact, but I believe in Jython it needs to be a separate
> declaration. Hope this helps.
>
> [1] https://github.com/apache/nifi/blob/master/nifi-api/src/
> main/java/org/apache/nifi/flowfile/FlowFile.java
> [2] https://funnifi.blogspot.com/2016/03/executescript-
> json-to-json-revisited_14.html
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Nov 2, 2017, at 12:53 PM, James McMahon <jsmcmahon3@gmail.com> wrote:
>
> In python, I can use the requests library to post content something like
> htis:
>
> import requests
> url="https://abc.test.org"
> files={'file':open('/somedir/myfile.txt','rb')}
> r = requests.post(url,files=files)
>
> If I am in a python stream callback, how can I read the flowfile payload
> in the same way that the open() reads its file from disk?
>
>
>

Mime
View raw message