nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: OnStopped annotation
Date Sat, 05 Mar 2016 23:54:29 GMT
Piere

Congratulations, you’ve hit one of the rarest cases in Java type system especially with
this one. Consider buying a lottery ticket ;)
The eureka moment was when I executed this code:
SetSNMP.class.getMethod("close").getDeclaringClass());
To my surprise the output was:
class org.apache.nifi.snmp.processors.SetSNMP

I would expect 'class org.apache.nifi.snmp.processors.AbstractSNMPProcessor’. And that is
when I remembered something from a long past.
When extending from parameterized class some times compiler feels there is a need to generate
a synthetic “bridge” method modeled after the one in super class and so it does, but without
annotations etc.
Anyway, you can easily see that by executing the following:

System.out.println(SetSNMP.class.getMethod("close").isBridge());
System.out.println(SetSNMP.class.getSuperclass().getMethod("close").isBridge());

The output is:

true
false

So, the bug is in ReflectionUtils  and I just filed it - https://issues.apache.org/jira/browse/NIFI-1595

For now I say go with the trick you’ve mentioned earlier (see below) so you can finalize
testing and hopefully the patch will be in before your PR will get merged (Ill try to fast
track it), so you can remove that code
@OnStopped
@Override
public void close() {
     super.close();
}

Cheers
Oleg

On Mar 5, 2016, at 4:48 PM, Oleg Zhurakousky <ozhurakousky@hortonworks.com<mailto:ozhurakousky@hortonworks.com>>
wrote:

Ok, I do see a problem and no I can’t explain it yet ;) Will figure it tomorrow.

Cheers
Oleg
On Mar 5, 2016, at 3:04 PM, Pierre Villard <pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com>>
wrote:

No problem Oleg, I had a lot to do this week as well.
You are correct, this is the issue I have on my side.

2016-03-05 20:46 GMT+01:00 Oleg Zhurakousky <ozhurakousky@hortonworks.com<mailto:ozhurakousky@hortonworks.com>>:

Pierre

Sorry, didn’t get a chance earlier. Just checked out your branch and
building to see what’s going on.
Just to confirm what I am looking for is @OnStopped is not getting invoked
in ‘AbstractSNMPProcessor’ whenever you try to stop the processor. Correct?

Cheers
Oleg
On Mar 5, 2016, at 2:28 PM, Pierre Villard <pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com>
<mailto:pierre.villard.fr@gmail.com>> wrote:

A remark : I don't think this is IDE/debug related since changes in the
properties of my processor are not taken into account (because of the
method close() not being called).

Besides, when overriding close() in the processor class, it is working as
expected :

 @OnStopped
 @Override
 public void close() {
     super.close();
 }

I don't know if this something helping to find out what's going on?
What do you suggest regarding a PR?

2016-03-03 18:15 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com<mailto:aldrinpiri@gmail.com><mailto:
aldrinpiri@gmail.com<mailto:aldrinpiri@gmail.com>>>:

Thanks for the info.  I'll try to orchestrate a similar setup when I get a
bit of free time later in hopes of recreating and diagnosing.  For some
context, I was operating on OS X with a local instance and using IntelliJ.

On Thu, Mar 3, 2016 at 11:55 AM, Pierre Villard <
pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com><mailto:pierre.villard.fr@gmail.com>
wrote:

I am connecting remotely to my instance on default port suggested in
bootstrap.conf (8000) and I am using Eclipse IDE.
Also tried in local following recommendations from Oleg with
nifi-ide-integration to directly launch Nifi from Eclipse: same result.

2016-03-03 17:46 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com<mailto:aldrinpiri@gmail.com>>:

Thanks for the info.  Are you doing all of this debugging local to that
instance (via tunnel/VNC/etc) or connecting remotely across the network
to
that instance?  Also, what IDE are you using to bind to the remote
session?

On Thu, Mar 3, 2016 at 10:02 AM, Pierre Villard <
pierre.villard.fr@gmail.com<mailto:pierre.villard.fr@gmail.com>
wrote:

That's a RedHat 7.2 VM running on AWS EC2 services.

2016-03-03 15:55 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:

Hmm... Java version should be okay.  What OS are you running on?

On Thu, Mar 3, 2016 at 4:04 AM, Pierre Villard <
pierre.villard.fr@gmail.com>
wrote:

Aldrin,

Thanks looking at this!

There is no additional change other than what is on the branch.
Just
to
be
sure, I set up a clean VM with nothing else on it, checked out
the
branch,
built it (with Maven 3.3.9, Java 1.8.0_74, mvn clean install
-DskipTests),
started Nifi with debug enabled, added SNMP Get processor, set
properties,
started it and stopped it. AbstractSNMPProcessor#close method
never
got
triggered.

Do you think I should try with another version of Java?




2016-03-02 22:58 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:

Pierre,

I did a build of the specified branch, enabled the remote
debugging
and
added a breakpoint within the AbstractSNMPProcessor#close
method
which
was
successfully triggered on stopping.  I additionally added a
breakpoint
within ReflectionUtils and was able to trace that path to the
same
#close
method.  Not discounting that there is an issue, but it appears
there
may
be additional factors to consider in diagnosing the root cause.

Are there any additional changes beyond the the branch you
shared
above
or
other libraries/NARs included in your environment?  Could you
also
please
specify the version of Java you are using?

On Wed, Mar 2, 2016 at 5:53 AM, Pierre Villard <
pierre.villard.fr@gmail.com>
wrote:

Sure, the branch is available here :
https://github.com/pvillard31/nifi/tree/NIFI-1537
Let me know if I can be of any help.

2016-03-02 11:48 GMT+01:00 Oleg Zhurakousky <
ozhurakousky@hortonworks.com
:

Pierre

Is your branch available to look at? May be a second pair
of
eyes
is
due
;)

Cheers
Oleg

On Mar 2, 2016, at 4:20 AM, Pierre Villard <
pierre.villard.fr@gmail.com>
wrote:

Mark,

I just checked and it seems to be OK. My processor is a
NAR
and
the
nifi-api.jar is not into the nar. Actually, I took the
AMQP
processor
as
example to build and integrate the SNMP one in Nifi, so
not
sure
to
see
why
it does not work.

2016-03-01 18:38 GMT+01:00 Mark Payne <
markap14@hotmail.com
:

Pierre,

This feels to me like it could potentially be a
classpath
issue.
The
nifi-api.jar file is on the
root class path (exists in $NIFI_HOME/lib). If there is
a
different
class
definition in your processor,
then it will not find the annotation.

If you are deploying your processor as a NAR, though,
the
maven
nar
plugin
should prevent you
from bundling the nifi-api.jar into the nar. It may be
worth
while
though
to poke around and make sure
that you don't have two copies of the OnStopped
annotation
class
defined
in the classpath, though.

Thanks
-Mark



On Mar 1, 2016, at 12:15 PM, Pierre Villard <
pierre.villard.fr@gmail.com>
wrote:

Nice! I used your nifi-ide-integration project to
launch
Nifi
from
my
IDE
and I reproduced the issue just by starting/stopping
the
processor
(with
a
breakpoint in ReflectionUtils). I'll try to find some
time
to
reproduce
it
in a unit test if this can help to find the issue.

Pierre

2016-03-01 13:56 GMT+01:00 Oleg Zhurakousky <
ozhurakousky@hortonworks.com>:

Pierre

You can simplify your interactive debugging and do it
right
from
IDE
https://github.com/olegz/nifi-ide-integration/
Just make sure that versioning in Gradle reflects
current
version.
I’ll update as well when I get a chance.

Oleg

On Mar 1, 2016, at 4:47 AM, Pierre Villard <
pierre.villard.fr@gmail.com
<mailto:pierre.villard.fr@gmail.com>> wrote:

No it is not in a unit test, I remotely attached my
Eclipse
in
debug
mode
to my deployed Nifi instance.
I can try to code a unit test and reproduce the issue
using
the
example
you
gave.

2016-02-29 18:22 GMT+01:00 Oleg Zhurakousky <
ozhurakousky@hortonworks.com
<mailto:ozhurakousky@hortonworks.com>>:

I meant could you share the test code (via github)

On Feb 29, 2016, at 12:18 PM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com<mailto:
ozhurakousky@hortonworks.com

wrote:

Ok, so you are invoking
‘quietlyInvokeMethodsWithAnnotations’
in
your
test code?
If so could you your test code where you invoke it? I
have a
hunch,
but
want to look before I speculate.

Cheers
Oleg
On Feb 29, 2016, at 11:58 AM, Pierre Villard <
pierre.villard.fr@gmail.com<mailto:
pierre.villard.fr@gmail.com

wrote:

I just wanted to test the processors with local SNMP
set-up
and
I
noticed
that modification of properties in my processor didn't
have
any
effect.
So I switched to debug, added a processor, started it,
and
stopped
it
just
after. Conclusion: my close() method is never called.
I correctly go through
quietlyInvokeMethodsWithAnnotations()
in
ReflectionUtils but since the method is not seen as
annotated,
the
close
method is not called.

Thanks,
Pierre

2016-02-28 22:24 GMT+01:00 Oleg Zhurakousky <
ozhurakousky@hortonworks.com<mailto:
ozhurakousky@hortonworks.com
:

I am puzzled as I can’t see how can it not work.
Are there steps to reproduce it? I am trying to read
into
your
initial
email and suspecting you were doing some sort of
testing,
so
want
to
make
sure I am doing the same thing. . . .

Oleg
On Feb 28, 2016, at 2:46 PM, Pierre Villard <
pierre.villard.fr@gmail.com<mailto:
pierre.villard.fr@gmail.com

wrote:

No I am not under testing framework, all my unit tests
are
OK. I
wanted
to
perform some additional tests and deployed Nifi with
the
new
processors.

You can find the method here [1] if you want to have a
look.
Thanks for your help.

[1]














https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243

2016-02-28 17:11 GMT+01:00 Oleg Zhurakousky <
ozhurakousky@hortonworks.com>:

Also, reading Aldrin’s response and assuming you are
using
Test
mocks I
would probably recommend to not use them for tests
that
require
full
lifecycle test of the component until we actually
improve
it.
Instead you can code straight agains FlowController
essentially
executing
as a full blown NiFi minus UI. Here is an example:














https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92

Cheers
Oleg

On Feb 28, 2016, at 11:07 AM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com<mailto:
ozhurakousky@hortonworks.com

wrote:

Pierre
Can you paste the method definition? Just want to look
at
the
signature
and see if there is something obvious

Sent from my iPhone

On Feb 28, 2016, at 10:26, Pierre Villard <
pierre.villard.fr@gmail.com
<mailto:pierre.villard.fr@gmail.com>> wrote:

Hi,

I am working on SNMP processors [1] and I'm almost
ready
for a
PR...
but I
have an issue I can't explain. In my processors, I
have
implemented
a
method close() with the @OnStopped annotation but it
seems
the
annotation
is not seen. When debugging and stopping my
processor, I
correctly
go
through quietlyInvokeMethodsWithAnnotations() in
ReflectionUtils
and
my
method close() appears without any annotation. I
guess I
am
missing
something simple. Any idea?

[1] https://issues.apache.org/jira/browse/NIFI-1537


























Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message