spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ash <and...@andrewash.com>
Subject Re: Updating docs for running on Mesos
Date Tue, 13 May 2014 08:30:25 GMT
Completely agree about preferring to link to the upstream project rather
than a company's -- the only reason I'm using mesosphere's now is that I
see no alternative from mesos.apache.org

I included instructions for both using Mesosphere's packages and building
from scratch in the PR: https://github.com/apache/spark/pull/756


On Tue, May 13, 2014 at 12:59 AM, Matei Zaharia <matei.zaharia@gmail.com>wrote:

> I’ll ask the Mesos folks about this. Unfortunately it might be tough to
> link only to a company’s builds; but we can perhaps include them in
> addition to instructions for building Mesos from Apache.
>
> Matei
>
> On May 12, 2014, at 11:55 PM, Gerard Maas <gerard.maas@gmail.com> wrote:
>
> > Andrew,
> >
> > Mesosphere has binary releases here:
> > http://mesosphere.io/downloads/
> >
> > (Anecdote: I actually burned a CPU building Mesos from source. No
> kidding -
> > it was coming, as the laptop was crashing from time to time, but the
> mesos
> > build was that one drop too much)
> >
> > kr, Gerard.
> >
> >
> >
> > On Tue, May 13, 2014 at 6:57 AM, Andrew Ash <andrew@andrewash.com>
> wrote:
> >
> >> As far as I know, the upstream doesn't release binaries, only source
> code.
> >> The downloads page <https://mesos.apache.org/downloads/> for 0.18.0
> only
> >> has a source tarball.  Is there a binary release somewhere from Mesos
> that
> >> I'm missing?
> >>
> >>
> >> On Sun, May 11, 2014 at 2:16 PM, Patrick Wendell <pwendell@gmail.com>
> >> wrote:
> >>
> >>> Andrew,
> >>>
> >>> Updating these docs would be great! I think this would be a welcome
> >> change.
> >>>
> >>> In terms of packaging, it would be good to mention the binaries
> >>> produced by the upstream project as well, in addition to Mesosphere.
> >>>
> >>> - Patrick
> >>>
> >>> On Thu, May 8, 2014 at 12:51 AM, Andrew Ash <andrew@andrewash.com>
> >> wrote:
> >>>> The docs for how to run Spark on Mesos have changed very little since
> >>>> 0.6.0, but setting it up is much easier now than then.  Does it make
> >>> sense
> >>>> to revamp with the below changes?
> >>>>
> >>>>
> >>>> You no longer need to build mesos yourself as pre-built versions are
> >>>> available from Mesosphere: http://mesosphere.io/downloads/
> >>>>
> >>>> And the instructions guide you towards compiling your own distribution
> >> of
> >>>> Spark, when you can use the prebuilt versions of Spark as well.
> >>>>
> >>>>
> >>>> I'd like to split that portion of the documentation into two sections,
> >> a
> >>>> build-from-scratch section and a use-prebuilt section.  The new
> outline
> >>>> would look something like this:
> >>>>
> >>>>
> >>>> *Running Spark on Mesos*
> >>>>
> >>>> Installing Mesos
> >>>> - using prebuilt (recommended)
> >>>> - pointer to mesosphere's packages
> >>>> - from scratch
> >>>> - (similar to current)
> >>>>
> >>>>
> >>>> Connecting Spark to Mesos
> >>>> - loading distribution into an accessible location
> >>>> - Spark settings
> >>>>
> >>>> Mesos Run Modes
> >>>> - (same as current)
> >>>>
> >>>> Running Alongside Hadoop
> >>>> - (trim this down)
> >>>>
> >>>>
> >>>>
> >>>> Does that work for people?
> >>>>
> >>>>
> >>>> Thanks!
> >>>> Andrew
> >>>>
> >>>>
> >>>> PS Basically all the same:
> >>>>
> >>>> http://spark.apache.org/docs/0.6.0/running-on-mesos.html
> >>>> http://spark.apache.org/docs/0.6.2/running-on-mesos.html
> >>>> http://spark.apache.org/docs/0.7.3/running-on-mesos.html
> >>>> http://spark.apache.org/docs/0.8.1/running-on-mesos.html
> >>>> http://spark.apache.org/docs/0.9.1/running-on-mesos.html
> >>>>
> >>>
> >>
> https://people.apache.org/~pwendell/spark-1.0.0-rc3-docs/running-on-mesos.html
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message