Yes, just reran with the following

(spark_build)root@ip-10-45-130-206:~/spark# export MAVEN_OPTS="-Xmx4096mb -XX:MaxPermSize=1024M -XX:ReservedCodeCacheSize=1024m"
(spark_build)root@ip-10-45-130-206:~/spark# build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package

and grepping for java


root       641  9.9  0.3 4411732 49040 pts/4   Sl+  17:35   0:01 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -server -Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m -Dzinc.home=/root/spark/build/zinc-0.3.5.3 -classpath /root/spark/build/zinc-0.3.5.3/lib/compiler-interface-sources.jar:/root/spark/build/zinc-0.3.5.3/lib/incremental-compiler.jar:/root/spark/build/zinc-0.3.5.3/lib/nailgun-server.jar:/root/spark/build/zinc-0.3.5.3/lib/sbt-interface.jar:/root/spark/build/zinc-0.3.5.3/lib/scala-compiler.jar:/root/spark/build/zinc-0.3.5.3/lib/scala-library.jar:/root/spark/build/zinc-0.3.5.3/lib/scala-reflect.jar:/root/spark/build/zinc-0.3.5.3/lib/zinc.jar com.typesafe.zinc.Nailgun 3030 0
root       687  226  2.0 1803664 312876 pts/4  Sl+  17:36   0:22 /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m -classpath /opt/anaconda/envs/spark_build/share/apache-maven-3.3.3/boot/plexus-classworlds-2.5.2.jar -Dclassworlds.conf=/opt/anaconda/envs/spark_build/share/apache-maven-3.3.3/bin/m2.conf -Dmaven.home=/opt/anaconda/envs/spark_build/share/apache-maven-3.3.3 -Dmaven.multiModuleProjectDirectory=/root/spark org.codehaus.plexus.classworlds.launcher.Launcher -DzincPort=3030 -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package

On Tue, Sep 8, 2015 at 1:14 PM, Sean Owen <sowen@cloudera.com> wrote:
MAVEN_OPTS shouldn't affect zinc as it's an unrelated application. You
can run "zinc -J-Xmx4g..." in general, but in the provided script,
ZINC_OPTS seems to be the equivalent, yes. It kind of looks like your
mvn process isn't getting any special memory args there. Is MAVEN_OPTS
really exported?

FWIW I use my own local mvn and zinc and it works fine.

On Tue, Sep 8, 2015 at 6:05 PM, Benjamin Zaitlen <quasiben@gmail.com> wrote:
> I'm running zinv while compiling.  It seems that MAVEN_OPTS doesn't really
> change much?  Or perhaps I'm misunderstanding something -- grepping for java
> i see
>
>> root     24355  102  8.8 4687376 1350724 pts/4 Sl   16:51  11:08
>> /usr/lib/jvm/java-7-openjdk-amd64/bin/java -server -Xmx2g
>> -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m
>> -Dzinc.home=/root/spark/build/zinc-0.3.5.3 -classpath
>> /root/spark/build/zinc-0.3.5.3/lib/compiler-interface-sources.jar:/root/spark/build/zinc-0.3.5.3/lib/incremental-compiler.jar:/root/spark/build/zinc-0.3.5.3/lib/nailgun-server.jar:/root/spark/build/zinc-0.3.5.3/lib/sbt-interface.jar:/root/spark/build/zinc-0.3.5.3/lib/scala-compiler.jar:/root/spark/build/zinc-0.3.5.3/lib/scala-library.jar:/root/spark/build/zinc-0.3.5.3/lib/scala-reflect.jar:/root/spark/build/zinc-0.3.5.3/lib/zinc.jar
>> com.typesafe.zinc.Nailgun 3030 0
>> root     25151 22.0  3.2 2269092 495276 pts/4  Sl+  16:53   1:56
>> /usr/lib/jvm/java-7-openjdk-amd64/bin/java -Xms256m -Xmx512m -classpath
>> /opt/anaconda/envs/spark_build/share/apache-maven-3.3.3/boot/plexus-classworlds-2.5.2.jar
>> -Dclassworlds.conf=/opt/anaconda/envs/spark_build/share/apache-maven-3.3.3/bin/m2.conf
>> -Dmaven.home=/opt/anaconda/envs/spark_build/share/apache-maven-3.3.3
>> -Dmaven.multiModuleProjectDirectory=/root/spark
>> org.codehaus.plexus.classworlds.launcher.Launcher -DzincPort=3030 clean
>> package -DskipTests -Pyarn -Phive -Phive-thriftserver -Phadoop-2.4
>> -Dhadoop.version=2.4.0
>
>
> So the heap size is still 2g even with MAVEN_OPTS set with 4g.  I noticed
> that within build/mvn _COMPILE_JVM_OPTS is set to 2g and this is what
> ZINC_OPTS is set to.
>
> --Ben
>
>
> On Tue, Sep 8, 2015 at 11:06 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> Do you run Zinc while compiling ?
>>
>> Cheers
>>
>> On Tue, Sep 8, 2015 at 7:56 AM, Benjamin Zaitlen <quasiben@gmail.com>
>> wrote:
>>>
>>> I'm still getting errors with 3g.  I've increase to 4g and I'll report
>>> back
>>>
>>> To be clear:
>>>
>>> export MAVEN_OPTS="-Xmx4g -XX:MaxPermSize=1024M
>>> -XX:ReservedCodeCacheSize=1024m"
>>>
>>>> [ERROR] GC overhead limit exceeded -> [Help 1]
>>>> [ERROR]
>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>>>> -e switch.
>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>>> [ERROR]
>>>> [ERROR] For more information about the errors and possible solutions,
>>>> please read the following articles:
>>>> [ERROR] [Help 1]
>>>> http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
>>>> + return 1
>>>> + exit 1
>>>
>>>
>>> On Tue, Sep 8, 2015 at 10:03 AM, Sean Owen <sowen@cloudera.com> wrote:
>>>>
>>>> It might need more memory in certain situations / running certain
>>>> tests. If 3gb works for your relatively full build, yes you can open a
>>>> PR to change any occurrences of lower recommendations to 3gb.
>>>>
>>>> On Tue, Sep 8, 2015 at 3:02 PM, Benjamin Zaitlen <quasiben@gmail.com>
>>>> wrote:
>>>> > Ah, right.  Should've caught that.
>>>> >
>>>> > The docs seem to recommend 2gb.  Should that be increased as well?
>>>> >
>>>> > --Ben
>>>> >
>>>> > On Tue, Sep 8, 2015 at 9:33 AM, Sean Owen <sowen@cloudera.com> wrote:
>>>> >>
>>>> >> It shows you there that Maven is out of memory. Give it more heap. I
>>>> >> use
>>>> >> 3gb.
>>>> >>
>>>> >> On Tue, Sep 8, 2015 at 1:53 PM, Benjamin Zaitlen <quasiben@gmail.com>
>>>> >> wrote:
>>>> >> > Hi All,
>>>> >> >
>>>> >> > I'm trying to build a distribution off of the latest in master and
>>>> >> > I
>>>> >> > keep
>>>> >> > getting errors on MQTT and the build fails.   I'm running the build
>>>> >> > on a
>>>> >> > m1.large which has 7.5 GB of RAM and no other major processes are
>>>> >> > running.
>>>> >> >
>>>> >> >> MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M
>>>> >> >> -XX:ReservedCodeCacheSize=512m"
>>>> >> >> ./make-distribution.sh  --name continuum-custom-spark-1.5 --tgz
>>>> >> >> -Pyarn
>>>> >> >> -Phive -Phive-thriftserver -Phadoop-2.4 -Dhadoop.version=2.4.0
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >> INFO] Spark Project GraphX ............................... SUCCESS
>>>> >> >> [
>>>> >> >> 33.345 s]
>>>> >> >> [INFO] Spark Project Streaming ............................
>>>> >> >> SUCCESS
>>>> >> >> [01:08
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project Catalyst .............................
>>>> >> >> SUCCESS
>>>> >> >> [01:39
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project SQL ..................................
>>>> >> >> SUCCESS
>>>> >> >> [02:06
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project ML Library ...........................
>>>> >> >> SUCCESS
>>>> >> >> [02:16
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project Tools ................................
>>>> >> >> SUCCESS [
>>>> >> >> 4.087 s]
>>>> >> >> [INFO] Spark Project Hive .................................
>>>> >> >> SUCCESS
>>>> >> >> [01:28
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project REPL .................................
>>>> >> >> SUCCESS [
>>>> >> >> 16.291 s]
>>>> >> >> [INFO] Spark Project YARN Shuffle Service .................
>>>> >> >> SUCCESS [
>>>> >> >> 13.671 s]
>>>> >> >> [INFO] Spark Project YARN .................................
>>>> >> >> SUCCESS [
>>>> >> >> 20.554 s]
>>>> >> >> [INFO] Spark Project Hive Thrift Server ...................
>>>> >> >> SUCCESS [
>>>> >> >> 14.332 s]
>>>> >> >> [INFO] Spark Project Assembly .............................
>>>> >> >> SUCCESS
>>>> >> >> [03:33
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project External Twitter .....................
>>>> >> >> SUCCESS [
>>>> >> >> 14.208 s]
>>>> >> >> [INFO] Spark Project External Flume Sink ..................
>>>> >> >> SUCCESS [
>>>> >> >> 11.535 s]
>>>> >> >> [INFO] Spark Project External Flume .......................
>>>> >> >> SUCCESS [
>>>> >> >> 19.010 s]
>>>> >> >> [INFO] Spark Project External Flume Assembly ..............
>>>> >> >> SUCCESS [
>>>> >> >> 5.210 s]
>>>> >> >> [INFO] Spark Project External MQTT ........................
>>>> >> >> FAILURE
>>>> >> >> [01:10
>>>> >> >> min]
>>>> >> >> [INFO] Spark Project External MQTT Assembly ...............
>>>> >> >> SKIPPED
>>>> >> >> [INFO] Spark Project External ZeroMQ ......................
>>>> >> >> SKIPPED
>>>> >> >> [INFO] Spark Project External Kafka .......................
>>>> >> >> SKIPPED
>>>> >> >> [INFO] Spark Project Examples .............................
>>>> >> >> SKIPPED
>>>> >> >> [INFO] Spark Project External Kafka Assembly ..............
>>>> >> >> SKIPPED
>>>> >> >> [INFO]
>>>> >> >>
>>>> >> >>
>>>> >> >> ------------------------------------------------------------------------
>>>> >> >> [INFO] BUILD FAILURE
>>>> >> >> [INFO]
>>>> >> >>
>>>> >> >>
>>>> >> >> ------------------------------------------------------------------------
>>>> >> >> [INFO] Total time: 22:55 min
>>>> >> >> [INFO] Finished at: 2015-09-07T22:42:57+00:00
>>>> >> >> [INFO] Final Memory: 240M/455M
>>>> >> >> [INFO]
>>>> >> >>
>>>> >> >>
>>>> >> >> ------------------------------------------------------------------------
>>>> >> >> [ERROR] GC overhead limit exceeded -> [Help 1]
>>>> >> >> [ERROR]
>>>> >> >> [ERROR] To see the full stack trace of the errors, re-run Maven
>>>> >> >> with
>>>> >> >> the
>>>> >> >> -e switch.
>>>> >> >> [ERROR] Re-run Maven using the -X switch to enable full debug
>>>> >> >> logging.
>>>> >> >> [ERROR]
>>>> >> >> [ERROR] For more information about the errors and possible
>>>> >> >> solutions,
>>>> >> >> please read the following articles:
>>>> >> >> [ERROR] [Help 1]
>>>> >> >> http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
>>>> >> >> + return 1
>>>> >> >> + exit 1
>>>> >> >
>>>> >> >
>>>> >> > Any thoughts would be extremely helpful.
>>>> >> >
>>>> >> > --Ben
>>>> >
>>>> >
>>>
>>>
>>
>