spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Losoi" <>
Subject Re: Development environments
Date Tue, 08 Oct 2013 13:29:08 GMT
> Hi Markus,

> have a look at the bottom of this wiki page:


> IntelliJ IDEA seems to be quite popular (that I am using myself)
> although Eclipse should work fine, too. There is another sbt plugin for
> generating Eclipse project files.

The IDE seems to work nicely, but what is the fastest way to build Spark? If
I make a change to the "core" module and choose "Make Module 'core'" from
the "Build" menu in IntelliJ Idea, then the IDE compiles the source code. To
create the "spark-assembly-0.8.0-incubating-hadoop1.0.4.jar" JAR file, I
have run "sbt assembly" on the command line. However, this takes an
impractically long time (843 s when I last ran it on my workstation with an
Intel Core 2 Quad Q9400 and 8 GB of RAM). Is there any faster way?

Best regards,
Markus Losoi (

View raw message