spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Burak Yavuz <brk...@gmail.com>
Subject Re: spark-submit --packages using different resolver
Date Sat, 03 Oct 2015 16:50:33 GMT
Hi Jerry,

The --packages feature doesn't support private repositories right now.
However, in the case of s3, maybe it might work. Could you please try using
the --repositories flag and provide the address:
`$ spark-submit --packages my:awesome:package --repositories
s3n://$aws_ak:$aws_sak@bucket/path/to/repo`

If that doesn't work, could you please file a JIRA?

Best,
Burak


On Thu, Oct 1, 2015 at 8:58 PM, Jerry Lam <chilinglam@gmail.com> wrote:

> Hi spark users and developers,
>
> I'm trying to use spark-submit --packages against private s3 repository.
> With sbt, I'm using fm-sbt-s3-resolver with proper aws s3 credentials. I
> wonder how can I add this resolver into spark-submit such that --packages
> can resolve dependencies from private repo?
>
> Thank you!
>
> Jerry
>

Mime
View raw message