buildr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Toulme <anto...@lunar-ocean.com>
Subject Re: Functional testing harness
Date Thu, 22 Jul 2010 21:03:48 GMT
Thanks for correcting me. Yes, integration tests it is.

And yes, we can't do anything serious without them.

On Thu, Jul 22, 2010 at 14:01, Assaf Arkin <assaf@labnotes.org> wrote:

> On Wed, Jul 21, 2010 at 10:09 PM, Antoine Toulme <antoine@lunar-ocean.com>wrote:
>
>> There actually will be crucial differences between specs and functional
>> tests. Mainly specs are a free for all to mock, squeeze, open wide open
>> internals to check the code as close as possible.
>>
>> Functional tests should run with the stock buildr, no changes.
>>
>> Your points are valid: we still need to run tests in a safe environment,
>> where we make sure to dispose of artifacts created.
>> So definitely a before and an after are in order.
>> The two tests added are peculiar - they don't create files just yet.
>>
>> However you think we're better off writing the Buildfile every time ? In
>> rake I see they kept the Buildfile in their own folders. This way I can
>> picture myself pointing people to "real" examples, real files they can see
>> and try for themselves to reproduce problems.
>>
>> Even if we keep the Buildfiles, we can still copy them dynamically in a
>> testbed. So your idea stands sound to me.
>>
>> At this point, I'd like to commit what I have, and refine to do a testbed
>> as
>> you suggest. Believe it or not, this is looking pretty exciting to me :)
>>
>
> I think you're looking at integration tests here.
>
> At the granularity of a Buildfile and running a full build (how else can
> you test packaging),  you're going to be testing multiple things in one go.
>
> Functional tests look at one function at a time. And you want to quickly
> run a small set of them to test the code as you're writing it (TDD), so
> functional tests contain a lot of mocking and stubbing.
>
> A lot of problems don't surface until you run end-to-end integration tests.
> You definitely want directory for each test, with the initial set of files.
> Besides being much easier to maintain, and like you said, can point people
> there to look at actual examples.
>
> Assaf
>
>
>>
>> On Wed, Jul 21, 2010 at 21:48, Peter Donald <peter@realityforge.org>
>> wrote:
>>
>> > Hi,
>> >
>> > On Thu, Jul 22, 2010 at 2:45 AM, Antoine Toulme <
>> antoine@lunar-ocean.com>
>> > wrote:
>> > > I'm looking for volunteers to take a quick look at the patch attached
>> to:
>> > > https://issues.apache.org/jira/browse/BUILDR-456
>> > >
>> > > And tell me what they think.
>> >
>> > As a first cut it looks not bad. I would expect to see a library of
>> > assertions built up over time to make it easy to check the output and
>> > results of the builds.
>> >
>> > One thing I would like to see is a consistency between the spec tests
>> > and the functional tests in how they setup the environment. In both
>> > cases you are typically going to need to delete then create  a scratch
>> > directory structure at start of test, configure (and populate) the
>> > maven repository. Both tests also may create some dirs and setup a
>> > build file (although in the spec tests it is done in memory while in
>> > the functional tests you are likely to be copying around files). At
>> > the end of the tests the scratch directory should be nuked.
>> >
>> > So extracting this into a common api would be great. It would also
>> > mean that it could be more easily used by outside people that are
>> > writing buildr plugins. I could imagine an api like
>> >
>> > # Put in test_helper/sepc_helper
>> > Buildr::Test.scratch_directory = File.basedir(__FILE__) + "/../tmp"
>> > Buildr::Test.clean_scratch_on_destroy = ENV["TEST_CLEAN_SCRATCH"].nil?
>> > ? true : ENV["TEST_CLEAN_SCRATCH"] == 'true'
>> >
>> > # called from setup
>> > Buildr::Test.init #
>> >
>> > # Called to copy artifact from local maven repo to maven repo in test
>> > Buildr::Test.install_artifact 'com.biz:my-widget:jar:2.0'
>> > Buildr::Test.install_artifact 'com.biz:my-other-widget:jar:2.0'
>> >
>> > # Example code to setup a functional test
>> > FileUtils.cp_r  File.basedir(__FILE__) + "/../functional/" +
>> > test_name, Buildr::Test.project_directory
>> >
>> > # called from teardown
>> > Buildr::Test.destroy
>> >
>> > Thoughts?
>> >
>> > --
>> > Cheers,
>> >
>> > Peter Donald
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message