spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imran Rashid <>
Subject Re: trouble with sbt building network-* projects?
Date Fri, 27 Feb 2015 19:31:23 GMT
well, perhaps I just need to learn to use maven better, but currently I
find sbt much more convenient for continuously running my tests.  I do use
zinc, but I'm looking for continuous testing.  This makes me think I need
sbt for that:

1) I really like that in sbt I can run "~test-only" (or whatever other pattern) and just leave that
running as I code, without having to go and explicitly trigger "mvn test"
and wait for the result.

2) I find sbt's handling of sub-projects much simpler (when it works).  I'm
trying to make changes to network/common & network/shuffle, which means I
have to keep cd'ing into network/common, run mvn install, then go back to
network/shuffle and run some other mvn command over there.  I don't want to
run mvn at the root project level, b/c I don't want to wait for it to
compile all the other projects when I just want to run tests in
network/common.  Even with incremental compiling, in my day-to-day coding I
want to entirely skip compiling sql, graphx, mllib etc. -- I have to switch
branches often enough that i end up triggering a full rebuild of those
projects even when I haven't touched them.

On Fri, Feb 27, 2015 at 1:14 PM, Ted Yu <> wrote:

> bq. to be able to run my tests in sbt, though, it makes the development
> iterations much faster.
> Was the preference for sbt due to long maven build time ?
> Have you started Zinc on your machine ?
> Cheers
> On Fri, Feb 27, 2015 at 11:10 AM, Imran Rashid <>
> wrote:
>> Has anyone else noticed very strange build behavior in the network-*
>> projects?
>> maven seems to the doing the right, but sbt is very inconsistent.
>> Sometimes when it builds network-shuffle it doesn't know about any of the
>> code in network-common.  Sometimes it will completely skip the java unit
>> tests.  And then some time later, it'll suddenly decide it knows about
>> some
>> more of the java unit tests.  Its not from a simple change, like touching
>> a
>> test file, or a file the test depends on -- nor a restart of sbt.  I am
>> pretty confused.
>> maven had issues when I tried to add scala code to network-common, it
>> would
>> compile the scala code but not make it available to java.  I'm working
>> around that by just coding in java anyhow.  I'd really like to be able to
>> run my tests in sbt, though, it makes the development iterations much
>> faster.
>> thanks,
>> Imran

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message