nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: OnStopped annotation
Date Sat, 05 Mar 2016 19:28:48 GMT
A remark : I don't think this is IDE/debug related since changes in the
properties of my processor are not taken into account (because of the
method close() not being called).

Besides, when overriding close() in the processor class, it is working as
expected :

    @OnStopped
    @Override
    public void close() {
        super.close();
    }

I don't know if this something helping to find out what's going on?
What do you suggest regarding a PR?

2016-03-03 18:15 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:

> Thanks for the info.  I'll try to orchestrate a similar setup when I get a
> bit of free time later in hopes of recreating and diagnosing.  For some
> context, I was operating on OS X with a local instance and using IntelliJ.
>
> On Thu, Mar 3, 2016 at 11:55 AM, Pierre Villard <
> pierre.villard.fr@gmail.com
> > wrote:
>
> > I am connecting remotely to my instance on default port suggested in
> > bootstrap.conf (8000) and I am using Eclipse IDE.
> > Also tried in local following recommendations from Oleg with
> > nifi-ide-integration to directly launch Nifi from Eclipse: same result.
> >
> > 2016-03-03 17:46 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
> >
> > > Thanks for the info.  Are you doing all of this debugging local to that
> > > instance (via tunnel/VNC/etc) or connecting remotely across the network
> > to
> > > that instance?  Also, what IDE are you using to bind to the remote
> > session?
> > >
> > > On Thu, Mar 3, 2016 at 10:02 AM, Pierre Villard <
> > > pierre.villard.fr@gmail.com
> > > > wrote:
> > >
> > > > That's a RedHat 7.2 VM running on AWS EC2 services.
> > > >
> > > > 2016-03-03 15:55 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
> > > >
> > > > > Hmm... Java version should be okay.  What OS are you running on?
> > > > >
> > > > > On Thu, Mar 3, 2016 at 4:04 AM, Pierre Villard <
> > > > > pierre.villard.fr@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Aldrin,
> > > > > >
> > > > > > Thanks looking at this!
> > > > > >
> > > > > > There is no additional change other than what is on the branch.
> > Just
> > > to
> > > > > be
> > > > > > sure, I set up a clean VM with nothing else on it, checked out
> the
> > > > > branch,
> > > > > > built it (with Maven 3.3.9, Java 1.8.0_74, mvn clean install
> > > > > -DskipTests),
> > > > > > started Nifi with debug enabled, added SNMP Get processor, set
> > > > > properties,
> > > > > > started it and stopped it. AbstractSNMPProcessor#close method
> never
> > > got
> > > > > > triggered.
> > > > > >
> > > > > > Do you think I should try with another version of Java?
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2016-03-02 22:58 GMT+01:00 Aldrin Piri <aldrinpiri@gmail.com>:
> > > > > >
> > > > > > > Pierre,
> > > > > > >
> > > > > > > I did a build of the specified branch, enabled the remote
> > debugging
> > > > and
> > > > > > > added a breakpoint within the AbstractSNMPProcessor#close
> method
> > > > which
> > > > > > was
> > > > > > > successfully triggered on stopping.  I additionally added
a
> > > > breakpoint
> > > > > > > within ReflectionUtils and was able to trace that path
to the
> > same
> > > > > #close
> > > > > > > method.  Not discounting that there is an issue, but it
appears
> > > there
> > > > > may
> > > > > > > be additional factors to consider in diagnosing the root
cause.
> > > > > > >
> > > > > > > Are there any additional changes beyond the the branch
you
> shared
> > > > above
> > > > > > or
> > > > > > > other libraries/NARs included in your environment?  Could
you
> > also
> > > > > please
> > > > > > > specify the version of Java you are using?
> > > > > > >
> > > > > > > On Wed, Mar 2, 2016 at 5:53 AM, Pierre Villard <
> > > > > > > pierre.villard.fr@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Sure, the branch is available here :
> > > > > > > > https://github.com/pvillard31/nifi/tree/NIFI-1537
> > > > > > > > Let me know if I can be of any help.
> > > > > > > >
> > > > > > > > 2016-03-02 11:48 GMT+01:00 Oleg Zhurakousky <
> > > > > > > ozhurakousky@hortonworks.com
> > > > > > > > >:
> > > > > > > >
> > > > > > > > > Pierre
> > > > > > > > >
> > > > > > > > > Is your branch available to look at? May be a
second pair
> of
> > > eyes
> > > > > is
> > > > > > > due
> > > > > > > > ;)
> > > > > > > > >
> > > > > > > > > Cheers
> > > > > > > > > Oleg
> > > > > > > > >
> > > > > > > > > > On Mar 2, 2016, at 4:20 AM, Pierre Villard
<
> > > > > > > > pierre.villard.fr@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Mark,
> > > > > > > > > >
> > > > > > > > > > I just checked and it seems to be OK. My
processor is a
> NAR
> > > and
> > > > > the
> > > > > > > > > > nifi-api.jar is not into the nar. Actually,
I took the
> AMQP
> > > > > > processor
> > > > > > > > as
> > > > > > > > > > example to build and integrate the SNMP
one in Nifi, so
> not
> > > > sure
> > > > > to
> > > > > > > see
> > > > > > > > > why
> > > > > > > > > > it does not work.
> > > > > > > > > >
> > > > > > > > > > 2016-03-01 18:38 GMT+01:00 Mark Payne <
> > markap14@hotmail.com
> > > >:
> > > > > > > > > >
> > > > > > > > > >> Pierre,
> > > > > > > > > >>
> > > > > > > > > >> This feels to me like it could potentially
be a
> classpath
> > > > issue.
> > > > > > The
> > > > > > > > > >> nifi-api.jar file is on the
> > > > > > > > > >> root class path (exists in $NIFI_HOME/lib).
If there is
> a
> > > > > > different
> > > > > > > > > class
> > > > > > > > > >> definition in your processor,
> > > > > > > > > >> then it will not find the annotation.
> > > > > > > > > >>
> > > > > > > > > >> If you are deploying your processor
as a NAR, though,
> the
> > > > maven
> > > > > > nar
> > > > > > > > > plugin
> > > > > > > > > >> should prevent you
> > > > > > > > > >> from bundling the nifi-api.jar into
the nar. It may be
> > worth
> > > > > while
> > > > > > > > > though
> > > > > > > > > >> to poke around and make sure
> > > > > > > > > >> that you don't have two copies of the
OnStopped
> annotation
> > > > class
> > > > > > > > defined
> > > > > > > > > >> in the classpath, though.
> > > > > > > > > >>
> > > > > > > > > >> Thanks
> > > > > > > > > >> -Mark
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >>> On Mar 1, 2016, at 12:15 PM, Pierre
Villard <
> > > > > > > > > pierre.villard.fr@gmail.com>
> > > > > > > > > >> wrote:
> > > > > > > > > >>>
> > > > > > > > > >>> Nice! I used your nifi-ide-integration
project to
> launch
> > > Nifi
> > > > > > from
> > > > > > > my
> > > > > > > > > IDE
> > > > > > > > > >>> and I reproduced the issue just
by starting/stopping
> the
> > > > > > processor
> > > > > > > > > (with
> > > > > > > > > >> a
> > > > > > > > > >>> breakpoint in ReflectionUtils).
I'll try to find some
> > time
> > > to
> > > > > > > > reproduce
> > > > > > > > > >> it
> > > > > > > > > >>> in a unit test if this can help
to find the issue.
> > > > > > > > > >>>
> > > > > > > > > >>> Pierre
> > > > > > > > > >>>
> > > > > > > > > >>> 2016-03-01 13:56 GMT+01:00 Oleg
Zhurakousky <
> > > > > > > > > >> ozhurakousky@hortonworks.com>:
> > > > > > > > > >>>
> > > > > > > > > >>>> Pierre
> > > > > > > > > >>>>
> > > > > > > > > >>>> You can simplify your interactive
debugging and do it
> > > right
> > > > > from
> > > > > > > IDE
> > > > > > > > > >>>> https://github.com/olegz/nifi-ide-integration/
> > > > > > > > > >>>> Just make sure that versioning
in Gradle reflects
> > current
> > > > > > version.
> > > > > > > > > >>>> I’ll update as well when I
get a chance.
> > > > > > > > > >>>>
> > > > > > > > > >>>> Oleg
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Mar 1, 2016, at 4:47 AM,
Pierre Villard <
> > > > > > > > > pierre.villard.fr@gmail.com
> > > > > > > > > >>>> <mailto:pierre.villard.fr@gmail.com>>
wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> No it is not in a unit test,
I remotely attached my
> > > Eclipse
> > > > in
> > > > > > > debug
> > > > > > > > > >> mode
> > > > > > > > > >>>> to my deployed Nifi instance.
> > > > > > > > > >>>> I can try to code a unit test
and reproduce the issue
> > > using
> > > > > the
> > > > > > > > > example
> > > > > > > > > >> you
> > > > > > > > > >>>> gave.
> > > > > > > > > >>>>
> > > > > > > > > >>>> 2016-02-29 18:22 GMT+01:00 Oleg
Zhurakousky <
> > > > > > > > > >> ozhurakousky@hortonworks.com
> > > > > > > > > >>>> <mailto:ozhurakousky@hortonworks.com>>:
> > > > > > > > > >>>>
> > > > > > > > > >>>> I meant could you share the
test code (via github)
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Feb 29, 2016, at 12:18 PM,
Oleg Zhurakousky <
> > > > > > > > > >>>> ozhurakousky@hortonworks.com<mailto:
> > > > > > ozhurakousky@hortonworks.com
> > > > > > > >>
> > > > > > > > > >> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> Ok, so you are invoking
> > > > ‘quietlyInvokeMethodsWithAnnotations’
> > > > > in
> > > > > > > > your
> > > > > > > > > >>>> test code?
> > > > > > > > > >>>> If so could you your test code
where you invoke it? I
> > > have a
> > > > > > > hunch,
> > > > > > > > > but
> > > > > > > > > >>>> want to look before I speculate.
> > > > > > > > > >>>>
> > > > > > > > > >>>> Cheers
> > > > > > > > > >>>> Oleg
> > > > > > > > > >>>> On Feb 29, 2016, at 11:58 AM,
Pierre Villard <
> > > > > > > > > >>>> pierre.villard.fr@gmail.com<mailto:
> > > > > pierre.villard.fr@gmail.com
> > > > > > >>
> > > > > > > > > wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> I just wanted to test the processors
with local SNMP
> > > set-up
> > > > > and
> > > > > > I
> > > > > > > > > >>>> noticed
> > > > > > > > > >>>> that modification of properties
in my processor didn't
> > > have
> > > > > any
> > > > > > > > > effect.
> > > > > > > > > >>>> So I switched to debug, added
a processor, started it,
> > and
> > > > > > stopped
> > > > > > > > it
> > > > > > > > > >>>> just
> > > > > > > > > >>>> after. Conclusion: my close()
method is never called.
> > > > > > > > > >>>> I correctly go through
> > > quietlyInvokeMethodsWithAnnotations()
> > > > > in
> > > > > > > > > >>>> ReflectionUtils but since the
method is not seen as
> > > > annotated,
> > > > > > the
> > > > > > > > > close
> > > > > > > > > >>>> method is not called.
> > > > > > > > > >>>>
> > > > > > > > > >>>> Thanks,
> > > > > > > > > >>>> Pierre
> > > > > > > > > >>>>
> > > > > > > > > >>>> 2016-02-28 22:24 GMT+01:00 Oleg
Zhurakousky <
> > > > > > > > > >>>> ozhurakousky@hortonworks.com<mailto:
> > > > > > ozhurakousky@hortonworks.com
> > > > > > > >>:
> > > > > > > > > >>>>
> > > > > > > > > >>>> I am puzzled as I can’t see
how can it not work.
> > > > > > > > > >>>> Are there steps to reproduce
it? I am trying to read
> > into
> > > > your
> > > > > > > > initial
> > > > > > > > > >>>> email and suspecting you were
doing some sort of
> > testing,
> > > so
> > > > > > want
> > > > > > > to
> > > > > > > > > >>>> make
> > > > > > > > > >>>> sure I am doing the same thing.
. . .
> > > > > > > > > >>>>
> > > > > > > > > >>>> Oleg
> > > > > > > > > >>>> On Feb 28, 2016, at 2:46 PM,
Pierre Villard <
> > > > > > > > > >>>> pierre.villard.fr@gmail.com<mailto:
> > > > > pierre.villard.fr@gmail.com
> > > > > > >>
> > > > > > > > > >>>> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> No I am not under testing framework,
all my unit tests
> > are
> > > > > OK. I
> > > > > > > > > >>>> wanted
> > > > > > > > > >>>> to
> > > > > > > > > >>>> perform some additional tests
and deployed Nifi with
> the
> > > new
> > > > > > > > > >>>> processors.
> > > > > > > > > >>>>
> > > > > > > > > >>>> You can find the method here
[1] if you want to have a
> > > look.
> > > > > > > > > >>>> Thanks for your help.
> > > > > > > > > >>>>
> > > > > > > > > >>>> [1]
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/pvillard31/nifi/blob/NIFI-1537/nifi-nar-bundles/nifi-snmp-bundle/nifi-snmp-processors/src/main/java/org/apache/nifi/snmp/processors/AbstractSNMPProcessor.java#L212-L243
> > > > > > > > > >>>>
> > > > > > > > > >>>> 2016-02-28 17:11 GMT+01:00 Oleg
Zhurakousky <
> > > > > > > > > >>>> ozhurakousky@hortonworks.com>:
> > > > > > > > > >>>>
> > > > > > > > > >>>> Also, reading Aldrin’s response
and assuming you are
> > using
> > > > > Test
> > > > > > > > > >>>> mocks I
> > > > > > > > > >>>> would probably recommend to
not use them for tests
> that
> > > > > require
> > > > > > > full
> > > > > > > > > >>>> lifecycle test of the component
until we actually
> > improve
> > > > it.
> > > > > > > > > >>>> Instead you can code straight
agains FlowController
> > > > > essentially
> > > > > > > > > >>>> executing
> > > > > > > > > >>>> as a full blown NiFi minus UI.
Here is an example:
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/nifi/pull/210/files#diff-7be646c38c5447f7824e444343633829R92
> > > > > > > > > >>>>
> > > > > > > > > >>>> Cheers
> > > > > > > > > >>>> Oleg
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Feb 28, 2016, at 11:07 AM,
Oleg Zhurakousky <
> > > > > > > > > >>>> ozhurakousky@hortonworks.com<mailto:
> > > > > > ozhurakousky@hortonworks.com
> > > > > > > >>
> > > > > > > > > >>>> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> Pierre
> > > > > > > > > >>>> Can you paste the method definition?
Just want to look
> > at
> > > > the
> > > > > > > > > >>>> signature
> > > > > > > > > >>>> and see if there is something
obvious
> > > > > > > > > >>>>
> > > > > > > > > >>>> Sent from my iPhone
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Feb 28, 2016, at 10:26, Pierre
Villard <
> > > > > > > > > >>>> pierre.villard.fr@gmail.com
> > > > > > > > > >>>> <mailto:pierre.villard.fr@gmail.com>>
wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>> Hi,
> > > > > > > > > >>>>
> > > > > > > > > >>>> I am working on SNMP processors
[1] and I'm almost
> ready
> > > > for a
> > > > > > > PR...
> > > > > > > > > >>>> but I
> > > > > > > > > >>>> have an issue I can't explain.
In my processors, I
> have
> > > > > > > implemented
> > > > > > > > a
> > > > > > > > > >>>> method close() with the @OnStopped
annotation but it
> > seems
> > > > the
> > > > > > > > > >>>> annotation
> > > > > > > > > >>>> is not seen. When debugging
and stopping my
> processor, I
> > > > > > correctly
> > > > > > > > go
> > > > > > > > > >>>> through quietlyInvokeMethodsWithAnnotations()
in
> > > > > ReflectionUtils
> > > > > > > and
> > > > > > > > > >>>> my
> > > > > > > > > >>>> method close() appears without
any annotation. I
> guess I
> > > am
> > > > > > > missing
> > > > > > > > > >>>> something simple. Any idea?
> > > > > > > > > >>>>
> > > > > > > > > >>>> [1] https://issues.apache.org/jira/browse/NIFI-1537
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message