nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: OnStopped annotation
Date Sun, 06 Mar 2016 12:33:36 GMT
Oleg

Great job! I would have never find this one! I didn't know about this
"bridge" feature in Java.
Thanks a lot for your help on this! I will proceed as suggested.

Pierre

2016-03-06 0:54 GMT+01:00 Oleg Zhurakousky <ozhurakousky@hortonworks.com>:

> Piere
>
> Congratulations, you’ve hit one of the rarest cases in Java type system
> especially with this one. Consider buying a lottery ticket ;)
> The eureka moment was when I executed this code:
> SetSNMP.class.getMethod("close").getDeclaringClass());
> To my surprise the output was:
> class org.apache.nifi.snmp.processors.SetSNMP
>
> I would expect 'class
> org.apache.nifi.snmp.processors.AbstractSNMPProcessor’. And that is when I
> remembered something from a long past.
> When extending from parameterized class some times compiler feels there is
> a need to generate a synthetic “bridge” method modeled after the one in
> super class and so it does, but without annotations etc.
> Anyway, you can easily see that by executing the following:
>
> System.out.println(SetSNMP.class.getMethod("close").isBridge());
>
> System.out.println(SetSNMP.class.getSuperclass().getMethod("close").isBridge());
>
> The output is:
>
> true
> false
>
> So, the bug is in ReflectionUtils  and I just filed it -
> https://issues.apache.org/jira/browse/NIFI-1595
>
> For now I say go with the trick you’ve mentioned earlier (see below) so
> you can finalize testing and hopefully the patch will be in before your PR
> will get merged (Ill try to fast track it), so you can remove that code
> @OnStopped
> @Override
> public void close() {
>      super.close();
> }
>
> Cheers
> Oleg
>
> On Mar 5, 2016, at 4:48 PM, Oleg Zhurakousky <ozhurakousky@hortonworks.com
> <mailto:ozhurakousky@hortonworks.com>> wrote:
>
> Ok, I do see a problem and no I can’t explain it yet ;) Will figure it
> tomorrow.
>
> Cheers
> Oleg
> On Mar 5, 2016, at 3:04 PM, Pierre Villard <pierre.villard.fr@gmail.com
> <mailto:pierre.villard.fr@gmail.com>> wrote:
>
> No problem Oleg, I had a lot to do this week as well.
> You are correct, this is the issue I have on my side.
>
> 2016-03-05 20:46 GMT+01:00 Oleg Zhurakousky <ozhurakousky@hortonworks.com
> <mailto:ozhurakousky@hortonworks.com>>:
>
> Pierre
>
> Sorry, didn’t get a chance earlier. Just checked out your branch and
> building to see what’s going on.
> Just to confirm what I am looking for is @OnStopped is not getting invoked
> in ‘AbstractSNMPProcessor’ whenever you try to stop the processor. Correct?
>
> Cheers
> Oleg
> On Mar 5, 2016, at 2:28 PM, Pierre Villard <pierre.villard.fr@gmail.com
> <mailto:pierre.villard.fr@gmail.com>
> <mailto:pierre.villard.fr@gmail.com>> wrote:
>
> A remark : I don't think this is IDE/debug related since changes in the
> properties of my processor are not taken into account (because of the
> method close() not being called).
>
> Besides, when overriding close() in the processor class, it is working as
> expected :
>
>  @OnStopped
>  @Override
>  public void close() {
>      super.close();
>  }
>
> I don't know if this something helping to find out what's going on?
> What do you suggest regarding a PR?
>
> 2016-03-03 18:15 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com<mailto:
> aldrinpiri@gmail.com><mailto:
> aldrinpiri@gmail.com<mailto:aldrinpiri@gmail.com>>>:
>
> Thanks for the info.  I'll try to orchestrate a similar setup when I get a
> bit of free time later in hopes of recreating and diagnosing.  For some
> context, I was operating on OS X with a local instance and using IntelliJ.
>
> On Thu, Mar 3, 2016 at 11:55 AM, Pierre Villard <
> pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com><mailto:
> pierre.villard.fr@gmail.com>
> wrote:
>
> I am connecting remotely to my instance on default port suggested in
> bootstrap.conf (8000) and I am using Eclipse IDE.
> Also tried in local following recommendations from Oleg with
> nifi-ide-integration to directly launch Nifi from Eclipse: same result.
>
> 2016-03-03 17:46 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com<mailto:
> aldrinpiri@gmail.com>>:
>
> Thanks for the info.  Are you doing all of this debugging local to that
> instance (via tunnel/VNC/etc) or connecting remotely across the network
> to
> that instance?  Also, what IDE are you using to bind to the remote
> session?
>
> On Thu, Mar 3, 2016 at 10:02 AM, Pierre Villard <
> pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com>
> wrote:
>
> That's a RedHat 7.2 VM running on AWS EC2 services.
>
> 2016-03-03 15:55 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
>
> Hmm... Java version should be okay.  What OS are you running on?
>
> On Thu, Mar 3, 2016 at 4:04 AM, Pierre Villard <
> pierre.villard.fr@gmail.com>
> wrote:
>
> Aldrin,
>
> Thanks looking at this!
>
> There is no additional change other than what is on the branch.
> Just
> to
> be
> sure, I set up a clean VM with nothing else on it, checked out
> the
> branch,
> built it (with Maven 3.3.9, Java 1.8.0_74, mvn clean install
> -DskipTests),
> started Nifi with debug enabled, added SNMP Get processor, set
> properties,
> started it and stopped it. AbstractSNMPProcessor#close method
> never
> got
> triggered.
>
> Do you think I should try with another version of Java?
>
>
>
>
> 2016-03-02 22:58 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
>
> Pierre,
>
> I did a build of the specified branch, enabled the remote
> debugging
> and
> added a breakpoint within the AbstractSNMPProcessor#close
> method
> which
> was
> successfully triggered on stopping.  I additionally added a
> breakpoint
> within ReflectionUtils and was able to trace that path to the
> same
> #close
> method.  Not discounting that there is an issue, but it appears
> there
> may
> be additional factors to consider in diagnosing the root cause.
>
> Are there any additional changes beyond the the branch you
> shared
> above
> or
> other libraries/NARs included in your environment?  Could you
> also
> please
> specify the version of Java you are using?
>
> On Wed, Mar 2, 2016 at 5:53 AM, Pierre Villard <
> pierre.villard.fr@gmail.com>
> wrote:
>
> Sure, the branch is available here :
> https://github.com/pvillard31/nifi/tree/NIFI-1537
> Let me know if I can be of any help.
>
> 2016-03-02 11:48 GMT+01:00 Oleg Zhurakousky <
> ozhurakousky@hortonworks.com
> :
>
> Pierre
>
> Is your branch available to look at? May be a second pair
> of
> eyes
> is
> due
> ;)
>
> Cheers
> Oleg
>
> On Mar 2, 2016, at 4:20 AM, Pierre Villard <
> pierre.villard.fr@gmail.com>
> wrote:
>
> Mark,
>
> I just checked and it seems to be OK. My processor is a
> NAR
> and
> the
> nifi-api.jar is not into the nar. Actually, I took the
> AMQP
> processor
> as
> example to build and integrate the SNMP one in Nifi, so
> not
> sure
> to
> see
> why
> it does not work.
>
> 2016-03-01 18:38 GMT+01:00 Mark Payne <
> markap14@hotmail.com
> :
>
> Pierre,
>
> This feels to me like it could potentially be a
> classpath
> issue.
> The
> nifi-api.jar file is on the
> root class path (exists in $NIFI_HOME/lib). If there is
> a
> different
> class
> definition in your processor,
> then it will not find the annotation.
>
> If you are deploying your processor as a NAR, though,
> the
> maven
> nar
> plugin
> should prevent you
> from bundling the nifi-api.jar into the nar. It may be
> worth
> while
> though
> to poke around and make sure
> that you don't have two copies of the OnStopped
> annotation
> class
> defined
> in the classpath, though.
>
> Thanks
> -Mark
>
>
>
> On Mar 1, 2016, at 12:15 PM, Pierre Villard <
> pierre.villard.fr@gmail.com>
> wrote:
>
> Nice! I used your nifi-ide-integration project to
> launch
> Nifi
> from
> my
> IDE
> and I reproduced the issue just by starting/stopping
> the
> processor
> (with
> a
> breakpoint in ReflectionUtils). I'll try to find some
> time
> to
> reproduce
> it
> in a unit test if this can help to find the issue.
>
> Pierre
>
> 2016-03-01 13:56 GMT+01:00 Oleg Zhurakousky <
> ozhurakousky@hortonworks.com>:
>
> Pierre
>
> You can simplify your interactive debugging and do it
> right
> from
> IDE
> https://github.com/olegz/nifi-ide-integration/
> Just make sure that versioning in Gradle reflects
> current
> version.
> I’ll update as well when I get a chance.
>
> Oleg
>
> On Mar 1, 2016, at 4:47 AM, Pierre Villard <
> pierre.villard.fr@gmail.com
> <mailto:pierre.villard.fr@gmail.com>> wrote:
>
> No it is not in a unit test, I remotely attached my
> Eclipse
> in
> debug
> mode
> to my deployed Nifi instance.
> I can try to code a unit test and reproduce the issue
> using
> the
> example
> you
> gave.
>
> 2016-02-29 18:22 GMT+01:00 Oleg Zhurakousky <
> ozhurakousky@hortonworks.com
> <mailto:ozhurakousky@hortonworks.com>>:
>
> I meant could you share the test code (via github)
>
> On Feb 29, 2016, at 12:18 PM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com<mailto:
> ozhurakousky@hortonworks.com
>
> wrote:
>
> Ok, so you are invoking
> ‘quietlyInvokeMethodsWithAnnotations’
> in
> your
> test code?
> If so could you your test code where you invoke it? I
> have a
> hunch,
> but
> want to look before I speculate.
>
> Cheers
> Oleg
> On Feb 29, 2016, at 11:58 AM, Pierre Villard <
> pierre.villard.fr@gmail.com<mailto:
> pierre.villard.fr@gmail.com
>
> wrote:
>
> I just wanted to test the processors with local SNMP
> set-up
> and
> I
> noticed
> that modification of properties in my processor didn't
> have
> any
> effect.
> So I switched to debug, added a processor, started it,
> and
> stopped
> it
> just
> after. Conclusion: my close() method is never called.
> I correctly go through
> quietlyInvokeMethodsWithAnnotations()
> in
> ReflectionUtils but since the method is not seen as
> annotated,
> the
> close
> method is not called.
>
> Thanks,
> Pierre
>
> 2016-02-28 22:24 GMT+01:00 Oleg Zhurakousky <
> ozhurakousky@hortonworks.com<mailto:
> ozhurakousky@hortonworks.com
> :
>
> I am puzzled as I can’t see how can it not work.
> Are there steps to reproduce it? I am trying to read
> into
> your
> initial
> email and suspecting you were doing some sort of
> testing,
> so
> want
> to
> make
> sure I am doing the same thing. . . .
>
> Oleg
> On Feb 28, 2016, at 2:46 PM, Pierre Villard <
> pierre.villard.fr@gmail.com<mailto:
> pierre.villard.fr@gmail.com
>
> wrote:
>
> No I am not under testing framework, all my unit tests
> are
> OK. I
> wanted
> to
> perform some additional tests and deployed Nifi with
> the
> new
> processors.
>
> You can find the method here [1] if you want to have a
> look.
> Thanks for your help.
>
> [1]
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243
>
> 2016-02-28 17:11 GMT+01:00 Oleg Zhurakousky <
> ozhurakousky@hortonworks.com>:
>
> Also, reading Aldrin’s response and assuming you are
> using
> Test
> mocks I
> would probably recommend to not use them for tests
> that
> require
> full
> lifecycle test of the component until we actually
> improve
> it.
> Instead you can code straight agains FlowController
> essentially
> executing
> as a full blown NiFi minus UI. Here is an example:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92
>
> Cheers
> Oleg
>
> On Feb 28, 2016, at 11:07 AM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com<mailto:
> ozhurakousky@hortonworks.com
>
> wrote:
>
> Pierre
> Can you paste the method definition? Just want to look
> at
> the
> signature
> and see if there is something obvious
>
> Sent from my iPhone
>
> On Feb 28, 2016, at 10:26, Pierre Villard <
> pierre.villard.fr@gmail.com
> <mailto:pierre.villard.fr@gmail.com>> wrote:
>
> Hi,
>
> I am working on SNMP processors [1] and I'm almost
> ready
> for a
> PR...
> but I
> have an issue I can't explain. In my processors, I
> have
> implemented
> a
> method close() with the @OnStopped annotation but it
> seems
> the
> annotation
> is not seen. When debugging and stopping my
> processor, I
> correctly
> go
> through quietlyInvokeMethodsWithAnnotations() in
> ReflectionUtils
> and
> my
> method close() appears without any annotation. I
> guess I
> am
> missing
> something simple. Any idea?
>
> [1] https://issues.apache.org/jira/browse/NIFI-1537
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message