spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Boesch <java...@gmail.com>
Subject Re: Building Spark with Pants
Date Tue, 03 Feb 2015 00:33:11 GMT
There is a significant investment in sbt and maven - and they are not at
all likely to be going away. A third build tool?  Note that there is also
the perspective of building within an IDE - which actually works presently
for sbt and with a little bit of tweaking with maven as well.

2015-02-02 16:25 GMT-08:00 Nicholas Chammas <nicholas.chammas@gmail.com>:

> Does anyone here have experience with Pants
> <http://pantsbuild.github.io/index.html> or interest in trying to build
> Spark with it?
>
> Pants has an interesting story. It was born at Twitter to help them build
> their Scala, Java, and Python projects as several independent components in
> one monolithic repo. (It was inspired by a similar build tool at Google
> called blaze.) The mix of languages and sub-projects at Twitter seems
> similar to the breakdown we have in Spark.
>
> Pants has an interesting take on how a build system should work, and
> Twitter and Foursquare (who use Pants as their primary build tool) claim it
> helps enforce better build hygiene and maintainability.
>
> Some relevant talks:
>
>    - Building Scala Hygienically with Pants
>    <https://www.youtube.com/watch?v=ukqke8iTuH0>
>    - The Pants Build Tool at Twitter
>    <
> https://engineering.twitter.com/university/videos/the-pants-build-tool-at-twitter
> >
>    - Getting Started with the Pants Build System: Why Pants?
>    <
> https://engineering.twitter.com/university/videos/getting-started-with-the-pants-build-system-why-pants
> >
>
> At some point I may take a shot at converting Spark to use Pants as an
> experiment and just see what it’s like.
>
> Nick
> ​
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message