spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Davidson <ilike...@gmail.com>
Subject Re: Easy one
Date Wed, 07 May 2014 00:32:13 GMT
If you're using standalone mode, you need to make sure the Spark Workers
know about the extra memory. This can be configured in spark-env.sh on the
workers as

export SPARK_WORKER_MEMORY=4g


On Tue, May 6, 2014 at 5:29 PM, Ian Ferreira <ianferreira@hotmail.com>wrote:

> Hi there,
>
> Why can’t I seem to kick the executor memory higher? See below from EC2
> deployment using m1.large
>
>
> And in the spark-env.sh
> export SPARK_MEM=6154m
>
>
> And in the spark context
> sconf.setExecutorEnv("spark.executor.memory", "4g”)
>
> Cheers
> - Ian
>
>

Mime
View raw message