spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szuromi Tamás <trom...@gmail.com>
Subject Re: Spark version for Mesos 0.27.0
Date Thu, 31 May 2018 09:30:46 GMT
I see it in the Serenity docs. Anyway, I guess you are able to use the
newest version of Spark with the Mesos 0.27 without any issues so don't
have to dispense newer Spark features and fixes.

Thodoris Zois <zois@ics.forth.gr> ezt írta (időpont: 2018. máj. 31., Cs,
11:22):

> Hello,
>
> The reason is that I want to make some tests and use the oversubscription
> feature of Mesos along with Spark. Intel and Mesosphere have built a
> project, called Serenity that actually measures the usage slack on each
> Mesos agent and returns resources to the cluster.
>
> Unfortunately, Serenity is not compatible with newer versions of Mesos...
>
> - Thodoris
>
>
> On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
>
> Hey,
>
> I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used
> with fine-grained scheduling and not dynamic allocation. Also, newer Spark
> versions should work with older mesos versions like 0.27.
> Why do you have Mesos 0.27 btw?
>
> cheers,
> Tamas
>
> Thodoris Zois <zois@ics.forth.gr> ezt írta (időpont: 2018. máj. 31., Cs,
> 1:56):
>
> Hello,
>
> I need Mesos 0.27 for specific purposes and unfortunately I can’t use a
> newer version. Did you find anything? Could it be Spark 1.6?
>
> Except that, from which version Spark supports dynamic allocation on
> Mesos?
>
> - Thodoris
>
> On 25 May 2018, at 16:06, Jacek Laskowski <jacek@japila.pl> wrote:
>
> Hi,
>
> Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml
> and see when the mesos dependency version changed. That'd give you the most
> precise answer. I think it could've been 1.5 or older.
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
> On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zois@ics.forth.gr> wrote:
>
> Hello,
>
> Could you please tell me which version of Spark works with Apache Mesos
>  version 0.27.0? (I cannot find anything on docs at github)
>
> Thank you very much,
> Thodoris Zois
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
>

Mime
View raw message