spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Davidson <ilike...@gmail.com>
Subject Re: Mesos not working on head of master
Date Fri, 29 Nov 2013 04:48:55 GMT
Hey Evan,

Sorry about the issue with the Mesos urls. I submitted the patch that broke
it originally -- at the time, I did not know about the zk:// urls mapping
to Mesos. I went ahead and started PR
#217<https://github.com/apache/incubator-spark/pull/217> to
correct this, which also includes a unit test for creating the schedulers
to hopefully avoid this type of issue in the future (at least at this most
basic level).

I cannot speak for #2, though, which seems to be the more serious issue.



On Wed, Nov 27, 2013 at 10:01 PM, Evan Chan <ev@ooyala.com> wrote:

> 1) zk:// URIs can no longer be passed to SparkContext as the first
> argument, as the MESOS_REGEX was changed to take only mesos:// URIs.
> This essentially renders Spark incompatible with real production Mesos
> setups.
>
> 2) Spinning up tasks on a Mesos cluster, all of them fail right away,
> with no log message visible.  This might possibly be me, but this was
> working with 0.8.0.
>
> I can submit a pull request for #1, but a better fix would be to have
> some kind of MesosSuite so that any kind of breakages can be caught
> systematically.   Maybe some kind of VM, but ideas are welcome.  What
> do you guys think?
>
> (I've cc'ed two folks from Mesosphere, esp on setting up a test suite)
>
> -Evan
>
> --
> --
> Evan Chan
> Staff Engineer
> ev@ooyala.com  |
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message