spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rarediel <>
Subject how to use spark.mesos.constraints
Date Wed, 02 Dec 2015 01:14:42 GMT
I am trying to add mesos constraints to my spark-submit command in my
marathon file I am setting it to spark.mesos.coarse=true.

Here is an example of a constraint I am trying to set.

 --conf spark.mesos.constraint=cpus:2 

I want to use the constraints to control the amount of executors are created
so I can control the total memory of my spark job.

I've tried many variations of resource constraints, but no matter which
resource or what number, range, etc. I do I always get the error "Initial
job has not accepted any resources; check your cluster UI...".  My cluster
has the available resources.  Is there any examples I can look at where
people use resource constraints?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message