The reason is that I want to make some tests and use the oversubscription feature of Mesos along with Spark. Intel and Mesosphere have built a project, called Serenity that actually measures the usage slack on each Mesos agent and returns resources to the cluster.
Unfortunately, Serenity is not compatible with newer versions of Mesos...
On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we used with fine-grained scheduling and not dynamic allocation. Also, newer Spark versions should work with older mesos versions like 0.27.
Why do you have Mesos 0.27 btw?
I need Mesos 0.27 for specific purposes and unfortunately I can’t use a newer version. Did you find anything? Could it be Spark 1.6?
Except that, from which version Spark supports dynamic allocation on Mesos?
Mesos 0.27.0?! That's been a while. I'd search for the changes to pom.xml and see when the mesos dependency version changed. That'd give you the most precise answer. I think it could've been 1.5 or older.