spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre B <>
Subject Access several s3 buckets, with credentials containing "/"
Date Fri, 05 Jun 2015 07:03:37 GMT
Hi list!

My problem is quite simple.
I need to access several S3 buckets, using different credentials.:
val c1 =
val c2 =
val c3 =

One/several of those AWS credentials might contain "/" in the private access
This is a known problem and from my research, the only ways to deal with
these "/" are:
1/ use environment variables to set the AWS credentials, then access the s3
buckets without specifying the credentials
2/ set the hadoop configuration to contain the the credentials.

However, none of these solutions allow me to access different buckets, with
different credentials.

Can anyone help me on this?



View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message