spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <>
Subject Re: RFC: Remove "HBaseTest" from examples?
Date Tue, 19 Apr 2016 17:34:03 GMT
On Tue, Apr 19, 2016 at 10:28 AM, Reynold Xin <> wrote:
> Yea in general I feel examples that bring in a large amount of dependencies
> should be outside Spark.

Another option to avoid the dependency problem is to not ship examples
in the distribution, and maybe create a separate tarball for them;
removing HBaseTest only solves one of the dependency problems. Since
we have examples for flume and kafka, for example, the Spark
distribution ends up shipping flume and kafka jars (and a bunch of
other things).

> On Tue, Apr 19, 2016 at 10:15 AM, Marcelo Vanzin <>
> wrote:
>> Hey all,
>> Two reasons why I think we should remove that from the examples:
>> - HBase now has Spark integration in its own repo, so that really
>> should be the template for how to use HBase from Spark, making that
>> example less useful, even misleading.
>> - It brings up a lot of extra dependencies that make the size of the
>> Spark distribution grow.
>> Any reason why we shouldn't drop that example?


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message