spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc2)
Date Mon, 09 Dec 2013 04:07:17 GMT
Well, 195 is sufficient to give you something that runs, but it doesn't run
the same way as Spark built/distributed by other means -- e.g., after 195
the package still uses something equivalent to the old `run` script instead
of the current `spark-class` way.


On Sun, Dec 8, 2013 at 8:02 PM, Patrick Wendell <pwendell@gmail.com> wrote:

> Hey Mark,
>
> What I'm asking is whether this patch is sufficient to have a working
> debian build in 0.8.1, or are there other outstanding issues to make
> it work? By working I mean, within the initial design that was
> contributed (with repl-bin) it works according to that approach.
>
> We can redesign this packaging in 0.9. That will require having a PR
> against Apache Spark, discussing, etc. But it doesn't need to be on
> the critical path for this release.
>
> - Patrick
>
> On Sun, Dec 8, 2013 at 7:54 PM, Mark Hamstra <mark@clearstorydata.com>
> wrote:
> > Whatever Debian package gets built has to work, so that's the first
> > requirement.  I don't know how to decide whether a change is acceptable
> in
> > 0.8 or has to wait until 0.9, but the 0.9 packaging should definitely
> > leverage the assembly sub-project, making repl-bin unnecessary.
> >
> >
> > On Sun, Dec 8, 2013 at 7:46 PM, Patrick Wendell <pwendell@gmail.com>
> wrote:
> >
> >> Looked into this a bit more - I think removing repl-bin is something
> >> we should wait until 0.9 to do, because we've published it to maven in
> >> 0.8.0 and people might expect it to be there in 0.8.1.
> >>
> >> Merging the directly referenced pull request (195) seems like a good
> >> idea though since it fixes a bug in the script.
> >>
> >> Is that what you are suggesting?
> >>
> >> - Patrick
> >>
> >> On Sun, Dec 8, 2013 at 7:30 PM, Patrick Wendell <pwendell@gmail.com>
> >> wrote:
> >> > Hey Mark - ya this would be good to get in.
> >> >
> >> > Does merging that particular PR put this in sufficient shape for the
> >> > 0.8.1 release or are there other open patches we need to look at?
> >> >
> >> > - Patrick
> >> >
> >> > On Sun, Dec 8, 2013 at 6:05 PM, Mark Hamstra <mark@clearstorydata.com
> >
> >> wrote:
> >> >> SPARK-962 should be resolved before release.  See also:
> >> >> https://github.com/apache/incubator-spark/pull/195
> >> >>
> >> >> With the references to the way I changed Debian packaging for
> >> ClearStory,
> >> >> we should be at least 90% of the way toward doing it right for
> Apache.
> >> >>
> >> >>
> >> >> On Sun, Dec 8, 2013 at 5:29 PM, Patrick Wendell <pwendell@gmail.com>
> >> wrote:
> >> >>
> >> >>> For my own part I'll give a +1 to this RC.
> >> >>>
> >> >>> On Sun, Dec 8, 2013 at 4:30 PM, Taka Shinagawa <
> taka.epsilon@gmail.com
> >> >
> >> >>> wrote:
> >> >>> > OK. I will post the entire output via separate email. I just
> upgraded
> >> >>> > Hadoop to 2.2.0 recently. So there might be something I need
to
> >> >>> > remove/clean up.
> >> >>> >
> >> >>> >
> >> >>> > On Sun, Dec 8, 2013 at 4:24 PM, Patrick Wendell <
> pwendell@gmail.com>
> >> >>> wrote:
> >> >>> >
> >> >>> >> Hey Take,
> >> >>> >>
> >> >>> >> Could you start a separate thread to debug your build
issue? In
> that
> >> >>> >> thread, could you paste the exact build command and entire
> output?
> >> The
> >> >>> log
> >> >>> >> you posted here suggests the first build detected hadoop
1.0.4
> not
> >> 2.2.0
> >> >>> >> based on the assembly file name it is logging.
> >> >>> >>
> >> >>> >> ---
> >> >>> >> sent from my phone
> >> >>> >> On Dec 8, 2013 4:13 PM, "Taka Shinagawa" <taka.epsilon@gmail.com
> >
> >> >>> wrote:
> >> >>> >>
> >> >>> >> > With Hadoop 2.2.0 (& Java 1.7.0_45) installed,
I'm having
> trouble
> >> >>> >> > completing the build process (sbt/sbt assembly) on
Macbook. The
> >> sbt
> >> >>> >> command
> >> >>> >> > hangs at the last step.
> >> >>> >> >
> >> >>> >> > ...
> >> >>> >> > ...
> >> >>> >> > [info] SHA-1: ce8275f5841002164c4305c912a2892ec7c1d395
> >> >>> >> > [info] Packaging
> >> >>> >> >
> >> >>> >> >
> >> >>> >>
> >> >>>
> >>
> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/tools/target/scala-2.9.3/spark-tools-assembly-0.8.1-incubating.jar
> >> >>> >> > ...
> >> >>> >> > [info] SHA-1: 0657a347240266230247693f265a5797d40c326a
> >> >>> >> > [info] Packaging
> >> >>> >> >
> >> >>> >> >
> >> >>> >>
> >> >>>
> >>
> /Users/taka/Documents/Spark/Releases/spark-0.8.1-incubating-rc2/assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop1.0.4.jar
> >> >>> >> > ...
> >> >>> >> > (hangs here)
> >> >>> >> > --------------------------
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > On another Macbook with Hadoop 1.1.1 (& Java
1.7.0_45)
> installed,
> >> I
> >> >>> was
> >> >>> >> > able to build it successfully.
> >> >>> >> > ..
> >> >>> >> > ..
> >> >>> >> > [info] SHA-1: 77109cd085bd4f0d2b601b3451b35b961d357534
> >> >>> >> > [info] Packaging
> >> >>> >> >
> >> >>> >> >
> >> >>> >>
> >> >>>
> >>
> /Users/tshinagawa/Documents/Spark/RCs/spark-0.8.1-incubating/examples/target/scala-2.9.3/spark-examples-assembly-0.8.1-incubating.jar
> >> >>> >> > ...
> >> >>> >> > [info] Done packaging.
> >> >>> >> > [success] Total time: 266 s, completed Dec 8, 2013
3:03:10 PM
> >> >>> >> > --------------------------
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > On Sun, Dec 8, 2013 at 12:41 PM, Patrick Wendell
<
> >> pwendell@gmail.com>
> >> >>> >> > wrote:
> >> >>> >> >
> >> >>> >> > > Please vote on releasing the following candidate
as Apache
> Spark
> >> >>> >> > > (incubating) version 0.8.1.
> >> >>> >> > >
> >> >>> >> > > The tag to be voted on is v0.8.1-incubating
(commit
> bf23794a):
> >> >>> >> > >
> >> >>> >> > >
> >> >>> >> >
> >> >>> >>
> >> >>>
> >>
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=e6ba91b5a7527316202797fc3dce469ff86cf203
> >> >>> >> > >
> >> >>> >> > > The release files, including signatures, digests,
etc can be
> >> found
> >> >>> at:
> >> >>> >> > >
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2/
> >> >>> >> > >
> >> >>> >> > > Release artifacts are signed with the following
key:
> >> >>> >> > > https://people.apache.org/keys/committer/pwendell.asc
> >> >>> >> > >
> >> >>> >> > > The staging repository for this release can
be found at:
> >> >>> >> > >
> >> >>>
> https://repository.apache.org/content/repositories/orgapachespark-024/
> >> >>> >> > >
> >> >>> >> > > The documentation corresponding to this release
can be found
> at:
> >> >>> >> > >
> >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc2-docs/
> >> >>> >> > >
> >> >>> >> > > For information about the contents of this release
see:
> >> >>> >> > > <attached> draft of release notes
> >> >>> >> > > <attached> draft of release credits
> >> >>> >> > >
> >> >>>
> https://github.com/apache/incubator-spark/blob/branch-0.8/CHANGES.txt
> >> >>> >> > >
> >> >>> >> > > Please vote on releasing this package as Apache
Spark
> >> >>> 0.8.1-incubating!
> >> >>> >> > >
> >> >>> >> > > The vote is open until Wednesday, December 11th
at 21:00 UTC
> and
> >> >>> >> > > passes if a majority of at least 3 +1 PPMC votes
are cast.
> >> >>> >> > >
> >> >>> >> > > [ ] +1 Release this package as Apache Spark
0.8.1-incubating
> >> >>> >> > > [ ] -1 Do not release this package because ...
> >> >>> >> > >
> >> >>> >> > > To learn more about Apache Spark, please see
> >> >>> >> > > http://spark.incubator.apache.org/
> >> >>> >> > >
> >> >>> >> >
> >> >>> >>
> >> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message