spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: Keep or remove Debian packaging in Spark?
Date Mon, 09 Feb 2015 15:31:09 GMT
>
> it sounds like nobody intends these to be used to actually deploy Spark


I wouldn't go quite that far.  What we have now can serve as useful input
to a deployment tool like Chef, but the user is then going to need to add
some customization or configuration within the context of that tooling to
get Spark installed just the way they want.  So it is not so much that the
current Debian packaging can't be used as that it has never really been
intended to be a completely finished product that a newcomer could, for
example, use to install Spark completely and quickly to Ubuntu and have a
fully-functional environment in which they could then run all of the
examples, tutorials, etc.

Getting to that level of packaging (and maintenance) is something that I'm
not sure we want to do since that is a better fit with Bigtop and the
efforts of Cloudera, Horton Works, MapR, etc. to distribute Spark.

On Mon, Feb 9, 2015 at 2:41 AM, Sean Owen <sowen@cloudera.com> wrote:

> This is a straw poll to assess whether there is support to keep and
> fix, or remove, the Debian packaging-related config in Spark.
>
> I see several oldish outstanding JIRAs relating to problems in the
> packaging:
>
> https://issues.apache.org/jira/browse/SPARK-1799
> https://issues.apache.org/jira/browse/SPARK-2614
> https://issues.apache.org/jira/browse/SPARK-3624
> https://issues.apache.org/jira/browse/SPARK-4436
> (and a similar idea about making RPMs)
> https://issues.apache.org/jira/browse/SPARK-665
>
> The original motivation seems related to Chef:
>
>
> https://issues.apache.org/jira/browse/SPARK-2614?focusedCommentId=14070908&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14070908
>
> Mark's recent comments cast some doubt on whether it is essential:
>
> https://github.com/apache/spark/pull/4277#issuecomment-72114226
>
> and in recent conversations I didn't hear dissent to the idea of removing
> this.
>
> Is this still useful enough to fix up? All else equal I'd like to
> start to walk back some of the complexity of the build, but I don't
> know how all-else-equal it is. Certainly, it sounds like nobody
> intends these to be used to actually deploy Spark.
>
> I don't doubt it's useful to someone, but can they maintain the
> packaging logic elsewhere?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message