spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject Re: [DISCUSS] Necessity of Maven *and* SBT Build in Spark
Date Wed, 26 Feb 2014 20:41:56 GMT
yes. the Build.scala file behaves like a configuration file mostly, but
because it is scala you can use the full power of a real language when
needed. also i found writing sbt plugins doable (but not easy).


On Feb 26, 2014 2:12 PM, "Sean Owen" <sowen@cloudera.com> wrote:

> I also favor Maven. I don't the the logic is "because it's common". As
> Sandy says, it's because of the things that brings: more plugins,
> easier to consume by more developers, etc. These are, however, just
> some reasons 'for', and have to be considered against the other pros
> and cons.
>
> The choice of build tools affects users in a way that choice of, say,
> version control does not.  As long as a usable pom pops out somehow,
> and artifacts are on Maven Central, I think all users will be happy.
> Whether SBT or Maven is the single source of truth is not that
> important (and yes there should be only one).
>
> What is the benefit to SBT? It seems like it's mostly replicating
> Maven functionality. I don't doubt it, just wondering. Is it just that
> it's in Scala too? They seem pretty equivalent.
>
> Don't underestimate the power of Maven plugins though... especially
> when trying to make all the release artifacts and push it! There are
> known incantations for this, hard won by years of anguish with Maven.
>
> My hunch is it will come down to whether it's easier to make the pom
> from SBT, or the SBT build from a pom.
> --
> Sean Owen | Director, Data Science | London
>
>
> On Wed, Feb 26, 2014 at 6:59 PM, Koert Kuipers <koert@tresata.com> wrote:
> > i dont buy the argument that we should use it because its the most
> common.
> >
> >
> > if all we would do is use what is most common then we should switch to
> > java, svn and maven
> >
> >
> > On Wed, Feb 26, 2014 at 1:38 PM, Mark Grover <
> grover.markgrover@gmail.com>wrote:
> >
> >> Hi Patrick,
> >> And, to pile on what Sandy said. In my opinion, it's definitely more
> than
> >> just a matter of convenience. My comment below applies both to
> distribution
> >> builders but also people who have their own internal "distributions" (a
> few
> >> examples of which we have already seen on this thread already).
> >>
> >> If one has to ensure consistent and harmonized versions of dependencies
> >> (whether they are being built as a part of the distribution e.g.
> zookeeper
> >> or pulled in transitively e.g. jersey), inheriting a root pom is the
> only
> >> sane way I know of doing so. It's really painful and error prone for a
> >> packager wanting to bump up jersey version for the entire stack, to
> have to
> >> bump up the version in a root pom for all maven projects but have to
> also
> >> go to ant's build properties file for all ant based projects and
> possibly
> >> sbt's build properties file to bump up the version there. Now, it was
> >> suggested that sbt can read such a pom file with use of a plugin and
> that
> >> would work for me but I personally don't think the other alternative of
> >> parsing out the pom file in scala would fly all that much.
> >>
> >> And then, of course, there is this subjective point of people being very
> >> familiar with maven as compared to sbt, it having a larger community
> base
> >> and there is something to be said for that.
> >>
> >> Mark
> >>
> >>
> >> On Wed, Feb 26, 2014 at 9:42 AM, Patrick Wendell <pwendell@gmail.com>
> >> wrote:
> >>
> >> > @mridul - As far as I know both Maven and Sbt use fairly similar
> >> > processes for building the assembly/uber jar. We actually used to
> >> > package spark with sbt and there were no specific issues we
> >> > encountered and AFAIK sbt respects versioning of transitive
> >> > dependencies correctly. Do you have a specific bug listing for sbt
> >> > that indicates something is broken?
> >> >
> >> > @sandy - It sounds like you are saying that the CDH build would be
> >> > easier with Maven because you can inherit the POM. However, is this
> >> > just a matter of convenience for packagers or would standardizing on
> >> > sbt limit capabilities in some way? I assume that it would just mean a
> >> > bit more manual work for packagers having to figure out how to set the
> >> > hadoop version in SBT and exclude certain dependencies. For instance,
> >> > what does CDH about other components like Impala that are not based on
> >> > Maven at all?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message