spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: why is it so slow to run sbt/sbt assembly in my machine?
Date Wed, 22 Jan 2014 19:27:04 GMT
It's a bug in sbt and/or the sbt-assembly plugin.  AFAIK, nobody has
isolated yet exactly what triggers the problem (only a minority of users
run into it), but there are workarounds and an eventual fix.  The fix
appears to be PR #266 <https://github.com/apache/incubator-spark/pull/266>.
 Until that is merged, the combination of assembly/assembly, tools/assembly
and examples/assembly will get you to the same state as running the
top-level assembly target, but a lot faster if you are running into the
slow assembly bug.  There is also an
assemble-deps<https://github.com/apache/incubator-spark/pull/66>that
can be used by Spark developers.


On Wed, Jan 22, 2014 at 11:15 AM, dachuan <hdc1112@gmail.com> wrote:

> Hi, community,
>
> I have run sbt/sbt assembly, but it is stuck at the following place for 30
> minutes.
>
> Does anybody know what's the root cause? my machine's memory size is 4GB.
>
> thanks,
> dachuan.
>
> [info] Packaging
> C:\cygwin\home\dachuan\incubator-spark\assembly\target\scala-2.10\spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop1.0.4.jar
> ...
> [info] Packaging
> C:\cygwin\home\dachuan\incubator-spark\examples\target\scala-2.10\spark-examples-assembly-0.9.0-incubating-SNAPSHOT.jar
> ...
> (STUCK HERE)
>
> --
> Dachuan Huang
> Cellphone: 614-390-7234
> 2015 Neil Avenue
> Ohio State University
> Columbus, Ohio
> U.S.A.
> 43210
>

Mime
View raw message