buildr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Assaf Arkin <as...@labnotes.org>
Subject Re: Functional testing harness
Date Thu, 22 Jul 2010 21:01:13 GMT
On Wed, Jul 21, 2010 at 10:09 PM, Antoine Toulme <antoine@lunar-ocean.com>wrote:

> There actually will be crucial differences between specs and functional
> tests. Mainly specs are a free for all to mock, squeeze, open wide open
> internals to check the code as close as possible.
>
> Functional tests should run with the stock buildr, no changes.
>
> Your points are valid: we still need to run tests in a safe environment,
> where we make sure to dispose of artifacts created.
> So definitely a before and an after are in order.
> The two tests added are peculiar - they don't create files just yet.
>
> However you think we're better off writing the Buildfile every time ? In
> rake I see they kept the Buildfile in their own folders. This way I can
> picture myself pointing people to "real" examples, real files they can see
> and try for themselves to reproduce problems.
>
> Even if we keep the Buildfiles, we can still copy them dynamically in a
> testbed. So your idea stands sound to me.
>
> At this point, I'd like to commit what I have, and refine to do a testbed
> as
> you suggest. Believe it or not, this is looking pretty exciting to me :)
>

I think you're looking at integration tests here.

At the granularity of a Buildfile and running a full build (how else can you
test packaging),  you're going to be testing multiple things in one go.

Functional tests look at one function at a time. And you want to quickly run
a small set of them to test the code as you're writing it (TDD), so
functional tests contain a lot of mocking and stubbing.

A lot of problems don't surface until you run end-to-end integration tests.
You definitely want directory for each test, with the initial set of files.
Besides being much easier to maintain, and like you said, can point people
there to look at actual examples.

Assaf


>
> On Wed, Jul 21, 2010 at 21:48, Peter Donald <peter@realityforge.org>
> wrote:
>
> > Hi,
> >
> > On Thu, Jul 22, 2010 at 2:45 AM, Antoine Toulme <antoine@lunar-ocean.com
> >
> > wrote:
> > > I'm looking for volunteers to take a quick look at the patch attached
> to:
> > > https://issues.apache.org/jira/browse/BUILDR-456
> > >
> > > And tell me what they think.
> >
> > As a first cut it looks not bad. I would expect to see a library of
> > assertions built up over time to make it easy to check the output and
> > results of the builds.
> >
> > One thing I would like to see is a consistency between the spec tests
> > and the functional tests in how they setup the environment. In both
> > cases you are typically going to need to delete then create  a scratch
> > directory structure at start of test, configure (and populate) the
> > maven repository. Both tests also may create some dirs and setup a
> > build file (although in the spec tests it is done in memory while in
> > the functional tests you are likely to be copying around files). At
> > the end of the tests the scratch directory should be nuked.
> >
> > So extracting this into a common api would be great. It would also
> > mean that it could be more easily used by outside people that are
> > writing buildr plugins. I could imagine an api like
> >
> > # Put in test_helper/sepc_helper
> > Buildr::Test.scratch_directory = File.basedir(__FILE__) + "/../tmp"
> > Buildr::Test.clean_scratch_on_destroy = ENV["TEST_CLEAN_SCRATCH"].nil?
> > ? true : ENV["TEST_CLEAN_SCRATCH"] == 'true'
> >
> > # called from setup
> > Buildr::Test.init #
> >
> > # Called to copy artifact from local maven repo to maven repo in test
> > Buildr::Test.install_artifact 'com.biz:my-widget:jar:2.0'
> > Buildr::Test.install_artifact 'com.biz:my-other-widget:jar:2.0'
> >
> > # Example code to setup a functional test
> > FileUtils.cp_r  File.basedir(__FILE__) + "/../functional/" +
> > test_name, Buildr::Test.project_directory
> >
> > # called from teardown
> > Buildr::Test.destroy
> >
> > Thoughts?
> >
> > --
> > Cheers,
> >
> > Peter Donald
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message