spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jamborta <>
Subject spark.driver.memory is not set (pyspark, 1.1.0)
Date Wed, 01 Oct 2014 16:26:14 GMT
Hi all,

I cannot figure out why this command is not setting the driver memory (it is
setting the executor memory):

    conf = (SparkConf()
                .set("spark.driver.memory", "1G")
                .set("spark.executor.memory", "1G")
                .set("spark.executor.instances", 2)
                .set("spark.executor.cores", 4))
    sc = SparkContext(conf=conf)

whereas if I run the spark console:
./bin/pyspark --driver-memory 1G

it sets it correctly. Seemingly they both generate the same commands in the

thanks a lot,

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message