spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: RFC: Remove "HBaseTest" from examples?
Date Tue, 19 Apr 2016 17:34:03 GMT
On Tue, Apr 19, 2016 at 10:28 AM, Reynold Xin <rxin@databricks.com> wrote:
> Yea in general I feel examples that bring in a large amount of dependencies
> should be outside Spark.

Another option to avoid the dependency problem is to not ship examples
in the distribution, and maybe create a separate tarball for them;
removing HBaseTest only solves one of the dependency problems. Since
we have examples for flume and kafka, for example, the Spark
distribution ends up shipping flume and kafka jars (and a bunch of
other things).

> On Tue, Apr 19, 2016 at 10:15 AM, Marcelo Vanzin <vanzin@cloudera.com>
> wrote:
>>
>> Hey all,
>>
>> Two reasons why I think we should remove that from the examples:
>>
>> - HBase now has Spark integration in its own repo, so that really
>> should be the template for how to use HBase from Spark, making that
>> example less useful, even misleading.
>>
>> - It brings up a lot of extra dependencies that make the size of the
>> Spark distribution grow.
>>
>> Any reason why we shouldn't drop that example?



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message