nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James McMahon <jsmcmah...@gmail.com>
Subject Re: File causes unexpected stoppage in play
Date Wed, 14 Dec 2016 20:08:44 GMT
I've never tried to create and deploy my own custom bundle but would like
to try #3 as you recommend Oleg. If you can offer guidance to that process
let me try that. Thank you. -Jim

On Wed, Dec 14, 2016 at 2:57 PM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com> wrote:

> James
>
> Not sure what kind of liberties you have with your environment, but here
> is what you can do outside of full upgrade.
>
> 1. You can download and drop the bundle (NAR) from NiFi 0.7 release into
> 0.6 release. That should work (just replace it in the ‘lib’ directory of
> NiFi home). However, this particular component is located in
> ’nifi-standard-nar’ which means it contains majority of the processors and
> controller services that come with NiFi. So while upgrading one component
> you will essentially upgrade almost all and I am not sure what it may lead
> to, but you can definitely try.
>
> 2.  You can certainly checkout the NiFi 0.6.1 source from the repo and
> apply the same patch on UpdateContent, build NAR and replace it as
> described in the step above, Even though you’ll still be replacing the
> entire NAR the changes are only going to be in that single component.
>
> 3.  You can create your own custom bundle (NAR), copy the UnpackContent
> code from 0.7 branch, rename the actual class to something different (e.g.,
> UnzipContent) and then deploy it as your own custom bundle and use it in
> your flow.
>
> 4.  You can also apply patch through fixing the byte code in flight but
> that would require you to write javaagent and be familiar with byte code
> manipulations frameworks. Not advisable. . .
>
> Honestly, aside from full upgrade the only viable option here is #3 since
> it gives you full control and complete isolation from every other component
> in NiFi.
>
> Let me know what you think so we can guide you thru the process.
>
> Cheers
> Oleg
>
> On Dec 14, 2016, at 2:18 PM, James McMahon <jsmcmahon3@gmail.com> wrote:
>
> Oleg, I am bound to NiFi 0.6.1 by enterprise application dependencies. So
> this fix will not be in my baseline if I understand you correctly. Let me
> ask you this: is there any way I can build this into my code baseline -
> either through a code mod and rebuild or as a custom plugin feature
> specific to my flow? Thanks very much for your help.
>
> On Wed, Dec 14, 2016 at 10:59 AM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com> wrote:
>
>> James
>>
>> Could you also let us know what version of NiFi you are using? The issue
>> with properly handling InvalidPathException was fixed in NiFi 0.7.0 as part
>> of https://issues.apache.org/jira/browse/NIFI-920
>> It essentially has this catch block:
>> } catch (final ProcessException | InvalidPathException e) {
>>      logger.error("Unable to unpack {} due to {}; routing to failure",
>> new Object[]{flowFile, e});
>>      session.transfer(flowFile, REL_FAILURE);
>>
>> So I am wondering if you are on the proviso release?
>> Cheers
>> Oleg
>>
>> On Dec 14, 2016, at 10:49 AM, Joe Witt <joe.witt@gmail.com> wrote:
>>
>> James,
>>
>> Can you please share the full log entry for that failure.  It is
>> possible the processor is not catching the exception properly and
>> routing the data to failure.  It might simply be letting the exception
>> loose thus the framework detects the issue and rollsback the session
>> and yields the processor.
>>
>> Likely an easy thing to fix in the processor but please do share the
>> full nifi-app.log entry for this.
>>
>> Thanks
>> Joe
>>
>> On Wed, Dec 14, 2016 at 10:47 AM, James McMahon <jsmcmahon3@gmail.com>
>> wrote:
>>
>> Hello. Am wondering if anyone knows how to overcome a challenge with
>> unmappable file characters? I have used a GetFile processor to bring a
>> large
>> number of zip files into a NiFi flow. All read in successfully. I try to
>> then use an UnpackContent processor to unzip the files to their individual
>> file members. Most work just fine. However there appears to be a file that
>> throws this error in UnpackContent:
>>
>> failed to process session due to java.nio.file.InvalidPathException:
>> Malformed input or input contains unmappable characters
>>
>> My processing stalls. Nothing else flows. What is the proper way to
>> configure the UnpackContent processor step so that it shuttle such files
>> off
>> to the side when it encounters them, and permits the other files waiting
>> in
>> queue to process? I do now have a "failure" path established for my
>> UnpackContent processor, but for some reason it does not send these
>> problem
>> files down that path. I suspect it may be because the zip files does
>> unpack
>> successfully but the underlying file(s) within the zip cause processing to
>> choke.
>>
>> How can I engineer a flow to overcome this challenge? Thanks in advance
>> for
>> your help.
>>
>>
>>
>>
>
>

Mime
View raw message