spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: RFC: Remove "HBaseTest" from examples?
Date Tue, 19 Apr 2016 17:35:58 GMT
bq. create a separate tarball for them

Probably another thread can be started for the above.
I am fine with it.

On Tue, Apr 19, 2016 at 10:34 AM, Marcelo Vanzin <vanzin@cloudera.com>
wrote:

> On Tue, Apr 19, 2016 at 10:28 AM, Reynold Xin <rxin@databricks.com> wrote:
> > Yea in general I feel examples that bring in a large amount of
> dependencies
> > should be outside Spark.
>
> Another option to avoid the dependency problem is to not ship examples
> in the distribution, and maybe create a separate tarball for them;
> removing HBaseTest only solves one of the dependency problems. Since
> we have examples for flume and kafka, for example, the Spark
> distribution ends up shipping flume and kafka jars (and a bunch of
> other things).
>
> > On Tue, Apr 19, 2016 at 10:15 AM, Marcelo Vanzin <vanzin@cloudera.com>
> > wrote:
> >>
> >> Hey all,
> >>
> >> Two reasons why I think we should remove that from the examples:
> >>
> >> - HBase now has Spark integration in its own repo, so that really
> >> should be the template for how to use HBase from Spark, making that
> >> example less useful, even misleading.
> >>
> >> - It brings up a lot of extra dependencies that make the size of the
> >> Spark distribution grow.
> >>
> >> Any reason why we shouldn't drop that example?
>
>
>
> --
> Marcelo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
View raw message