spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From satish saley <satishsale...@gmail.com>
Subject System memory 186646528 must be at least 4.718592E8.
Date Fri, 13 May 2016 19:47:06 GMT
Hello,
I am running
https://github.com/apache/spark/blob/branch-1.6/examples/src/main/python/pi.py
example,
but facing following exception

What is the unit of memory pointed out in the error?

Following are configs

                    --master

                    local[*]

                    --deploy-mode

                    client

                    --name

                    PysparkExample

                    --py-files

                    py4j-0.9-src.zip,pyspark.zip,

                    --verbose


pi.py/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value

py4j.protocol.Py4JJavaError: An error occurred while calling
None.org.apache.spark.api.java.JavaSparkContext.

: java.lang.IllegalArgumentException: System memory 186646528 must be at
least 4.718592E8. Please use a larger heap size.

        at
org.apache.spark.memory.UnifiedMemoryManager$.getMaxMemory(UnifiedMemoryManager.scala:193)

        at
org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:175)

        at org.apache.spark.SparkEnv$.create(SparkEnv.scala:354)

        at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193)

        at
org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288)

        at org.apache.spark.SparkContext.<init>(SparkContext.scala:457)

        at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)

        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)

        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234)

        at
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)

        at py4j.Gateway.invoke(Gateway.java:214)

        at
py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79)

        at
py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68)

        at py4j.GatewayConnection.run(GatewayConnection.java:209)

        at java.lang.Thread.run(Thread.java:745)

Mime
View raw message