spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com>
Subject Re: Join highly skewed datasets
Date Sun, 28 Jun 2015 20:33:57 GMT
How can i import this pre-built spark into my application via maven as i
want to use the block join API.

On Sun, Jun 28, 2015 at 1:31 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepujain@gmail.com> wrote:

> I ran this w/o maven options
>
> ./make-distribution.sh  --tgz -Phadoop-2.4 -Pyarn  -Phive
> -Phive-thriftserver
>
> I got this spark-1.4.0-bin-2.4.0.tgz in the same working directory.
>
> I hope this is built with 2.4.x hadoop as i did specify -P
>
> On Sun, Jun 28, 2015 at 1:10 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepujain@gmail.com>
> wrote:
>
>>  ./make-distribution.sh  --tgz --*mvn* "-Phadoop-2.4 -Pyarn
>> -Dhadoop.version=2.4.0 -Phive -Phive-thriftserver -DskipTests clean package"
>>
>>
>> or
>>
>>
>>  ./make-distribution.sh  --tgz --*mvn* -Phadoop-2.4 -Pyarn
>> -Dhadoop.version=2.4.0 -Phive -Phive-thriftserver -DskipTests clean package"
>> ​Both fail with
>>
>> + echo -e 'Specify the Maven command with the --mvn flag'
>>
>> Specify the Maven command with the --mvn flag
>>
>> + exit -1
>>
>
>
>
> --
> Deepak
>
>


-- 
Deepak

Mime
View raw message