spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Build Changes for SBT Users
Date Mon, 06 Jan 2014 00:24:38 GMT
Why is it not possible? You always update the script; just can't update
scripts for released versions.




On Sat, Jan 4, 2014 at 9:07 PM, Patrick Wendell <pwendell@gmail.com> wrote:

> I agree TD - I was just saying that Reynold's proposal that we could
> update the release post-hoc is unfortunately not possible.
>
> On Sat, Jan 4, 2014 at 7:13 PM, Tathagata Das
> <tathagata.das1565@gmail.com> wrote:
> > Patrick, that is right. All we are trying to ensure is to make a
> > "best-effort" attempt to make it smooth for a new user. The script will
> try
> > its best to automatically install / download sbt for the user. The
> fallback
> > will be that the user will have to install sbt on their own. If the URL
> > happens to change and our script fails to automatically download, then we
> > are *no worse* than not providing the script at all.
> >
> > TD
> >
> >
> > On Sat, Jan 4, 2014 at 7:06 PM, Patrick Wendell <pwendell@gmail.com>
> wrote:
> >
> >> Reynold the issue is releases are immutable and we expect them to be
> >> downloaded for several years after the release date.
> >>
> >> On Sat, Jan 4, 2014 at 5:57 PM, Xuefeng Wu <benewu@gmail.com> wrote:
> >> > Sound reasonable.  But I think few installed sbt even it is easy to
> >> install.  I think can provide this tricky script in online document,
> user
> >> could download this script to install sbt independence. Sound like a yet
> >> another brew install sbt?
> >> > :)
> >> >
> >> > Yours, Xuefeng Wu 吴雪峰 敬上
> >> >
> >> >> On 2014年1月5日, at 上午2:56, Patrick Wendell <pwendell@gmail.com>
wrote:
> >> >>
> >> >> We thought about this but elected not to do this for a few reasons.
> >> >>
> >> >> 1. Some people build from machines that do not have internet access
> >> >> for security reasons and retrieve dependency from internal nexus
> >> >> repositories. So having a build dependency that relies on internet
> >> >> downloads is not desirable.
> >> >>
> >> >> 2. It's a hard to ensure stability of a particular URL in perpetuity.
> >> >> This is why maven central and other mirror networks exist. Keep in
> >> >> mind that we can't change the release code ever once we release it,
> >> >> and if something changed about the particular URL it could break the
> >> >> build.
> >> >>
> >> >> - Patrick
> >> >>
> >> >>> On Sat, Jan 4, 2014 at 9:34 AM, Andrew Ash <andrew@andrewash.com>
> >> wrote:
> >> >>> +1 on bundling a script similar to that one
> >> >>>
> >> >>>
> >> >>>> On Sat, Jan 4, 2014 at 4:48 AM, Holden Karau <holden@pigscanfly.ca
> >
> >> wrote:
> >> >>>>
> >> >>>> Could we ship a shell script which downloads the sbt jar if
not
> >> present
> >> >>>> (like for example
> https://github.com/holdenk/slashem/blob/master/sbt)?
> >> >>>>
> >> >>>>
> >> >>>> On Sat, Jan 4, 2014 at 12:02 AM, Patrick Wendell <
> pwendell@gmail.com>
> >> >>>> wrote:
> >> >>>>
> >> >>>>> Hey All,
> >> >>>>>
> >> >>>>> Due to an ASF requirement, we recently merged a patch which
> removes
> >> >>>>> the sbt jar from the build. This is necessary because we
aren't
> >> >>>>> allowed to distributed binary artifacts with our source
packages.
> >> >>>>>
> >> >>>>> This means that instead of building Spark with "sbt/sbt
XXX",
> you'll
> >> >>>>> need to have sbt yourself and just run "sbt XXX" from within
the
> >> Spark
> >> >>>>> directory. This is similar to the maven build, where we
expect
> users
> >> >>>>> already have maven installed.
> >> >>>>>
> >> >>>>> You can download sbt at http://www.scala-sbt.org/. It's
okay to
> just
> >> >>>>> download the most recent version of sbt, since sbt knows
how to
> fetch
> >> >>>>> other versions of itself and will always use the one we
specify in
> >> our
> >> >>>>> build file to compile spark.
> >> >>>>>
> >> >>>>> - Patrick
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> Cell : 425-233-8271
> >> >>>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message