nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Skora <jsk...@gmail.com>
Subject Re: "Integration" style Unit Tests
Date Wed, 14 Oct 2015 18:46:04 GMT
I agree with you that these shouldn't run as normal unit tests.  But, I'm
worried that it means this functionality will not be regularly tested as
external APIs change.  Mocking an external API doesn't validate much, if I
don't understand the external resource I'll probably make the same mistakes
in the mock as I do in the client, so testing against the real external
resource is the only way to know things work.

Perhaps a set of test scripts are needed to automate groups of @ignored
tests that share an external dependency such as an AWS script, a Hadoop
script, a Kafka script, etc?

I have never used JUnit's filtering capability, but it might be possible to
use that in combination with a Maven profile to automate those test groups
instead of requiring scripts be maintained.

Just brainstorming.

Regards,
JoeS

On Mon, Oct 12, 2015 at 8:22 PM, Joe Witt <joe.witt@gmail.com> wrote:

> Dan,
>
> Yeah this is a good housekeeping item to bring up.  My view here is
> that B is the only answer.  Unit tests should not be calling out to
> external services - period.
>
> Thanks
> Joe
>
> On Mon, Oct 12, 2015 at 8:17 PM, dan bress <danbress@gmail.com> wrote:
> > Devs,
> >    While working on integrating and testing the work Yu did for
> > NIFI-774/DeleteS3Object, I noticed that a few of the unit tests for the
> > processors in that AWS bundle(Put and Fetch S3) actually interact with S3
> > directly, and were marked as @Ignore.  If I wanted to un @Ignore them and
> > actually run them I needed to set up an AWS account, then copy the
> > credentials into an aws-credentials.properties file and put that in my
> home
> > directory to get the tests to pass.  This dashed my hopes of a relatively
> > simple merge and turned it into a bit of work for me.  I'm not blaming Yu
> > or anyone for this, just wanted to open up a discussion on better ways of
> > solving this.
> >
> > Problem:
> > @Ignore'd tests don't get run, probably ever.  Why?  Because running them
> > is a pain in the butt, I agree with NIFI-438[1] "If tests could talk they
> > would say don't @Ignore me".  I appreciate that there are special
> > circumstances for using this, but it would probably benefit everyone if
> we
> > sure we used it only for the truly special circumstances.
> >
> >
> > Solutions:
> > a. Run these tests using the failsafe plugin[2] instead of the surefire
> > plugin.  This way they get run every time, and if they fail that
> > information gets reported but it does not stop the build.
> > b. Mock out the service(I appreciate that this may not always be
> possible)
> > c. Provide instructions somewhere so that someone with no experience with
> > these processors/tests can run them.
> >
> > Anyone have thoughts on this?
> > My votes would be B, and if that is not possible A and C.
> >
> > Thanks,
> > Dan
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-438
> > [2] https://maven.apache.org/surefire/maven-failsafe-plugin/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message