spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: trouble with sbt building network-* projects?
Date Fri, 27 Feb 2015 21:54:54 GMT
bq. I have to keep cd'ing into network/common, run mvn install, then go
back to network/shuffle and run some other mvn command over there.

Yeah - been through this.

Having continuous testing for maven would be nice.

On Fri, Feb 27, 2015 at 11:31 AM, Imran Rashid <irashid@cloudera.com> wrote:

> well, perhaps I just need to learn to use maven better, but currently I
> find sbt much more convenient for continuously running my tests.  I do use
> zinc, but I'm looking for continuous testing.  This makes me think I need
> sbt for that:
> http://stackoverflow.com/questions/11347633/is-there-a-java-continuous-testing-plugin-for-maven
>
> 1) I really like that in sbt I can run "~test-only
> com.foo.bar.SomeTestSuite" (or whatever other pattern) and just leave that
> running as I code, without having to go and explicitly trigger "mvn test"
> and wait for the result.
>
> 2) I find sbt's handling of sub-projects much simpler (when it works).
> I'm trying to make changes to network/common & network/shuffle, which means
> I have to keep cd'ing into network/common, run mvn install, then go back to
> network/shuffle and run some other mvn command over there.  I don't want to
> run mvn at the root project level, b/c I don't want to wait for it to
> compile all the other projects when I just want to run tests in
> network/common.  Even with incremental compiling, in my day-to-day coding I
> want to entirely skip compiling sql, graphx, mllib etc. -- I have to switch
> branches often enough that i end up triggering a full rebuild of those
> projects even when I haven't touched them.
>
>
>
>
>
> On Fri, Feb 27, 2015 at 1:14 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
>> bq. to be able to run my tests in sbt, though, it makes the development
>> iterations much faster.
>>
>> Was the preference for sbt due to long maven build time ?
>> Have you started Zinc on your machine ?
>>
>> Cheers
>>
>> On Fri, Feb 27, 2015 at 11:10 AM, Imran Rashid <irashid@cloudera.com>
>> wrote:
>>
>>> Has anyone else noticed very strange build behavior in the network-*
>>> projects?
>>>
>>> maven seems to the doing the right, but sbt is very inconsistent.
>>> Sometimes when it builds network-shuffle it doesn't know about any of the
>>> code in network-common.  Sometimes it will completely skip the java unit
>>> tests.  And then some time later, it'll suddenly decide it knows about
>>> some
>>> more of the java unit tests.  Its not from a simple change, like
>>> touching a
>>> test file, or a file the test depends on -- nor a restart of sbt.  I am
>>> pretty confused.
>>>
>>>
>>> maven had issues when I tried to add scala code to network-common, it
>>> would
>>> compile the scala code but not make it available to java.  I'm working
>>> around that by just coding in java anyhow.  I'd really like to be able to
>>> run my tests in sbt, though, it makes the development iterations much
>>> faster.
>>>
>>> thanks,
>>> Imran
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message