spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aureliano Buendia <buendia...@gmail.com>
Subject Re: WARN ClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory
Date Thu, 09 Jan 2014 03:07:29 GMT
The strange thing is that spark examples work fine, but when I include a
spark example in my jar and deploy it, I get this error for the very same
example:

WARN ClusterScheduler: Initial job has not accepted any resources; check
your cluster UI to ensure that workers are registered and have sufficient
memory

My jar is deployed to master and then to workers by spark-ec2/copy-dir. Why
would including the example in my jar cause this error?



On Thu, Jan 9, 2014 at 12:41 AM, Aureliano Buendia <buendia360@gmail.com>wrote:

> Could someone explain how SPARK_MEM, SPARK_WORKER_MEMORY and
> spark.executor.memory should be related so that this non helpful error
> doesn't occur?
>
> Maybe there are more env and java config variable about memory that I'm
> missing.
>
> By the way, that bit of the error asking to check the web UI, it's just
> redundant. The UI is of no help.
>
>
> On Wed, Jan 8, 2014 at 4:31 PM, Aureliano Buendia <buendia360@gmail.com>wrote:
>
>> Hi,
>>
>>
>> My spark cluster is not able to run a job due to this warning:
>>
>> WARN ClusterScheduler: Initial job has not accepted any resources; check
>> your cluster UI to ensure that workers are registered and have sufficient
>> memory
>>
>> The workers have these status:
>>
>> ALIVE 2 (0 Used)6.3 GB (0.0 B Used) So there must be plenty of memory
>> available despite the warning message. I'm using default spark config, is
>> there a config parameter that needs changing for this to work?
>>
>
>

Mime
View raw message