nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: OnStopped annotation
Date Sat, 05 Mar 2016 21:48:09 GMT
Ok, I do see a problem and no I can’t explain it yet ;) Will figure it tomorrow.

Cheers
Oleg
> On Mar 5, 2016, at 3:04 PM, Pierre Villard <pierre.villard.fr@gmail.com> wrote:
> 
> No problem Oleg, I had a lot to do this week as well.
> You are correct, this is the issue I have on my side.
> 
> 2016-03-05 20:46 GMT+01:00 Oleg Zhurakousky <ozhurakousky@hortonworks.com>:
> 
>> Pierre
>> 
>> Sorry, didn’t get a chance earlier. Just checked out your branch and
>> building to see what’s going on.
>> Just to confirm what I am looking for is @OnStopped is not getting invoked
>> in ‘AbstractSNMPProcessor’ whenever you try to stop the processor. Correct?
>> 
>> Cheers
>> Oleg
>> On Mar 5, 2016, at 2:28 PM, Pierre Villard <pierre.villard.fr@gmail.com
>> <mailto:pierre.villard.fr@gmail.com>> wrote:
>> 
>> A remark : I don't think this is IDE/debug related since changes in the
>> properties of my processor are not taken into account (because of the
>> method close() not being called).
>> 
>> Besides, when overriding close() in the processor class, it is working as
>> expected :
>> 
>>   @OnStopped
>>   @Override
>>   public void close() {
>>       super.close();
>>   }
>> 
>> I don't know if this something helping to find out what's going on?
>> What do you suggest regarding a PR?
>> 
>> 2016-03-03 18:15 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com<mailto:
>> aldrinpiri@gmail.com>>:
>> 
>> Thanks for the info.  I'll try to orchestrate a similar setup when I get a
>> bit of free time later in hopes of recreating and diagnosing.  For some
>> context, I was operating on OS X with a local instance and using IntelliJ.
>> 
>> On Thu, Mar 3, 2016 at 11:55 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com>
>> wrote:
>> 
>> I am connecting remotely to my instance on default port suggested in
>> bootstrap.conf (8000) and I am using Eclipse IDE.
>> Also tried in local following recommendations from Oleg with
>> nifi-ide-integration to directly launch Nifi from Eclipse: same result.
>> 
>> 2016-03-03 17:46 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
>> 
>> Thanks for the info.  Are you doing all of this debugging local to that
>> instance (via tunnel/VNC/etc) or connecting remotely across the network
>> to
>> that instance?  Also, what IDE are you using to bind to the remote
>> session?
>> 
>> On Thu, Mar 3, 2016 at 10:02 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com
>> wrote:
>> 
>> That's a RedHat 7.2 VM running on AWS EC2 services.
>> 
>> 2016-03-03 15:55 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
>> 
>> Hmm... Java version should be okay.  What OS are you running on?
>> 
>> On Thu, Mar 3, 2016 at 4:04 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com>
>> wrote:
>> 
>> Aldrin,
>> 
>> Thanks looking at this!
>> 
>> There is no additional change other than what is on the branch.
>> Just
>> to
>> be
>> sure, I set up a clean VM with nothing else on it, checked out
>> the
>> branch,
>> built it (with Maven 3.3.9, Java 1.8.0_74, mvn clean install
>> -DskipTests),
>> started Nifi with debug enabled, added SNMP Get processor, set
>> properties,
>> started it and stopped it. AbstractSNMPProcessor#close method
>> never
>> got
>> triggered.
>> 
>> Do you think I should try with another version of Java?
>> 
>> 
>> 
>> 
>> 2016-03-02 22:58 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
>> 
>> Pierre,
>> 
>> I did a build of the specified branch, enabled the remote
>> debugging
>> and
>> added a breakpoint within the AbstractSNMPProcessor#close
>> method
>> which
>> was
>> successfully triggered on stopping.  I additionally added a
>> breakpoint
>> within ReflectionUtils and was able to trace that path to the
>> same
>> #close
>> method.  Not discounting that there is an issue, but it appears
>> there
>> may
>> be additional factors to consider in diagnosing the root cause.
>> 
>> Are there any additional changes beyond the the branch you
>> shared
>> above
>> or
>> other libraries/NARs included in your environment?  Could you
>> also
>> please
>> specify the version of Java you are using?
>> 
>> On Wed, Mar 2, 2016 at 5:53 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com>
>> wrote:
>> 
>> Sure, the branch is available here :
>> https://github.com/pvillard31/nifi/tree/NIFI-1537
>> Let me know if I can be of any help.
>> 
>> 2016-03-02 11:48 GMT+01:00 Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com
>> :
>> 
>> Pierre
>> 
>> Is your branch available to look at? May be a second pair
>> of
>> eyes
>> is
>> due
>> ;)
>> 
>> Cheers
>> Oleg
>> 
>> On Mar 2, 2016, at 4:20 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com>
>> wrote:
>> 
>> Mark,
>> 
>> I just checked and it seems to be OK. My processor is a
>> NAR
>> and
>> the
>> nifi-api.jar is not into the nar. Actually, I took the
>> AMQP
>> processor
>> as
>> example to build and integrate the SNMP one in Nifi, so
>> not
>> sure
>> to
>> see
>> why
>> it does not work.
>> 
>> 2016-03-01 18:38 GMT+01:00 Mark Payne <
>> markap14@hotmail.com
>> :
>> 
>> Pierre,
>> 
>> This feels to me like it could potentially be a
>> classpath
>> issue.
>> The
>> nifi-api.jar file is on the
>> root class path (exists in $NIFI_HOME/lib). If there is
>> a
>> different
>> class
>> definition in your processor,
>> then it will not find the annotation.
>> 
>> If you are deploying your processor as a NAR, though,
>> the
>> maven
>> nar
>> plugin
>> should prevent you
>> from bundling the nifi-api.jar into the nar. It may be
>> worth
>> while
>> though
>> to poke around and make sure
>> that you don't have two copies of the OnStopped
>> annotation
>> class
>> defined
>> in the classpath, though.
>> 
>> Thanks
>> -Mark
>> 
>> 
>> 
>> On Mar 1, 2016, at 12:15 PM, Pierre Villard <
>> pierre.villard.fr@gmail.com>
>> wrote:
>> 
>> Nice! I used your nifi-ide-integration project to
>> launch
>> Nifi
>> from
>> my
>> IDE
>> and I reproduced the issue just by starting/stopping
>> the
>> processor
>> (with
>> a
>> breakpoint in ReflectionUtils). I'll try to find some
>> time
>> to
>> reproduce
>> it
>> in a unit test if this can help to find the issue.
>> 
>> Pierre
>> 
>> 2016-03-01 13:56 GMT+01:00 Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com>:
>> 
>> Pierre
>> 
>> You can simplify your interactive debugging and do it
>> right
>> from
>> IDE
>> https://github.com/olegz/nifi-ide-integration/
>> Just make sure that versioning in Gradle reflects
>> current
>> version.
>> I’ll update as well when I get a chance.
>> 
>> Oleg
>> 
>> On Mar 1, 2016, at 4:47 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com
>> <mailto:pierre.villard.fr@gmail.com>> wrote:
>> 
>> No it is not in a unit test, I remotely attached my
>> Eclipse
>> in
>> debug
>> mode
>> to my deployed Nifi instance.
>> I can try to code a unit test and reproduce the issue
>> using
>> the
>> example
>> you
>> gave.
>> 
>> 2016-02-29 18:22 GMT+01:00 Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com
>> <mailto:ozhurakousky@hortonworks.com>>:
>> 
>> I meant could you share the test code (via github)
>> 
>> On Feb 29, 2016, at 12:18 PM, Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com<mailto:
>> ozhurakousky@hortonworks.com
>> 
>> wrote:
>> 
>> Ok, so you are invoking
>> ‘quietlyInvokeMethodsWithAnnotations’
>> in
>> your
>> test code?
>> If so could you your test code where you invoke it? I
>> have a
>> hunch,
>> but
>> want to look before I speculate.
>> 
>> Cheers
>> Oleg
>> On Feb 29, 2016, at 11:58 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com<mailto:
>> pierre.villard.fr@gmail.com
>> 
>> wrote:
>> 
>> I just wanted to test the processors with local SNMP
>> set-up
>> and
>> I
>> noticed
>> that modification of properties in my processor didn't
>> have
>> any
>> effect.
>> So I switched to debug, added a processor, started it,
>> and
>> stopped
>> it
>> just
>> after. Conclusion: my close() method is never called.
>> I correctly go through
>> quietlyInvokeMethodsWithAnnotations()
>> in
>> ReflectionUtils but since the method is not seen as
>> annotated,
>> the
>> close
>> method is not called.
>> 
>> Thanks,
>> Pierre
>> 
>> 2016-02-28 22:24 GMT+01:00 Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com<mailto:
>> ozhurakousky@hortonworks.com
>> :
>> 
>> I am puzzled as I can’t see how can it not work.
>> Are there steps to reproduce it? I am trying to read
>> into
>> your
>> initial
>> email and suspecting you were doing some sort of
>> testing,
>> so
>> want
>> to
>> make
>> sure I am doing the same thing. . . .
>> 
>> Oleg
>> On Feb 28, 2016, at 2:46 PM, Pierre Villard <
>> pierre.villard.fr@gmail.com<mailto:
>> pierre.villard.fr@gmail.com
>> 
>> wrote:
>> 
>> No I am not under testing framework, all my unit tests
>> are
>> OK. I
>> wanted
>> to
>> perform some additional tests and deployed Nifi with
>> the
>> new
>> processors.
>> 
>> You can find the method here [1] if you want to have a
>> look.
>> Thanks for your help.
>> 
>> [1]
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243
>> 
>> 2016-02-28 17:11 GMT+01:00 Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com>:
>> 
>> Also, reading Aldrin’s response and assuming you are
>> using
>> Test
>> mocks I
>> would probably recommend to not use them for tests
>> that
>> require
>> full
>> lifecycle test of the component until we actually
>> improve
>> it.
>> Instead you can code straight agains FlowController
>> essentially
>> executing
>> as a full blown NiFi minus UI. Here is an example:
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92
>> 
>> Cheers
>> Oleg
>> 
>> On Feb 28, 2016, at 11:07 AM, Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com<mailto:
>> ozhurakousky@hortonworks.com
>> 
>> wrote:
>> 
>> Pierre
>> Can you paste the method definition? Just want to look
>> at
>> the
>> signature
>> and see if there is something obvious
>> 
>> Sent from my iPhone
>> 
>> On Feb 28, 2016, at 10:26, Pierre Villard <
>> pierre.villard.fr@gmail.com
>> <mailto:pierre.villard.fr@gmail.com>> wrote:
>> 
>> Hi,
>> 
>> I am working on SNMP processors [1] and I'm almost
>> ready
>> for a
>> PR...
>> but I
>> have an issue I can't explain. In my processors, I
>> have
>> implemented
>> a
>> method close() with the @OnStopped annotation but it
>> seems
>> the
>> annotation
>> is not seen. When debugging and stopping my
>> processor, I
>> correctly
>> go
>> through quietlyInvokeMethodsWithAnnotations() in
>> ReflectionUtils
>> and
>> my
>> method close() appears without any annotation. I
>> guess I
>> am
>> missing
>> something simple. Any idea?
>> 
>> [1] https://issues.apache.org/jira/browse/NIFI-1537
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 

Mime
View raw message