spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imran Rashid <iras...@cloudera.com>
Subject Re: trouble with sbt building network-* projects?
Date Fri, 27 Feb 2015 19:31:23 GMT
well, perhaps I just need to learn to use maven better, but currently I
find sbt much more convenient for continuously running my tests.  I do use
zinc, but I'm looking for continuous testing.  This makes me think I need
sbt for that:
http://stackoverflow.com/questions/11347633/is-there-a-java-continuous-testing-plugin-for-maven

1) I really like that in sbt I can run "~test-only
com.foo.bar.SomeTestSuite" (or whatever other pattern) and just leave that
running as I code, without having to go and explicitly trigger "mvn test"
and wait for the result.

2) I find sbt's handling of sub-projects much simpler (when it works).  I'm
trying to make changes to network/common & network/shuffle, which means I
have to keep cd'ing into network/common, run mvn install, then go back to
network/shuffle and run some other mvn command over there.  I don't want to
run mvn at the root project level, b/c I don't want to wait for it to
compile all the other projects when I just want to run tests in
network/common.  Even with incremental compiling, in my day-to-day coding I
want to entirely skip compiling sql, graphx, mllib etc. -- I have to switch
branches often enough that i end up triggering a full rebuild of those
projects even when I haven't touched them.





On Fri, Feb 27, 2015 at 1:14 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> bq. to be able to run my tests in sbt, though, it makes the development
> iterations much faster.
>
> Was the preference for sbt due to long maven build time ?
> Have you started Zinc on your machine ?
>
> Cheers
>
> On Fri, Feb 27, 2015 at 11:10 AM, Imran Rashid <irashid@cloudera.com>
> wrote:
>
>> Has anyone else noticed very strange build behavior in the network-*
>> projects?
>>
>> maven seems to the doing the right, but sbt is very inconsistent.
>> Sometimes when it builds network-shuffle it doesn't know about any of the
>> code in network-common.  Sometimes it will completely skip the java unit
>> tests.  And then some time later, it'll suddenly decide it knows about
>> some
>> more of the java unit tests.  Its not from a simple change, like touching
>> a
>> test file, or a file the test depends on -- nor a restart of sbt.  I am
>> pretty confused.
>>
>>
>> maven had issues when I tried to add scala code to network-common, it
>> would
>> compile the scala code but not make it available to java.  I'm working
>> around that by just coding in java anyhow.  I'd really like to be able to
>> run my tests in sbt, though, it makes the development iterations much
>> faster.
>>
>> thanks,
>> Imran
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message