hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15672) add s3guard CLI command to generate session keys for an assumed role
Date Sat, 03 Nov 2018 16:13:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674092#comment-16674092

Steve Loughran commented on HADOOP-15672:

or better, hadoop tool CLI supports declaration of df file for loading into UGI, if it doesn't
do this already, 

hadoop fs -ls --tokens ~/tokens.bin s3a://bucket1/

> add s3guard CLI command to generate session keys for an assumed role
> --------------------------------------------------------------------
>                 Key: HADOOP-15672
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15672
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Priority: Minor
> the aws cli [get-session-token|https://docs.aws.amazon.com/cli/latest/reference/sts/get-session-token.html]
can generate the keys for short-lived session.
> I'd like something similar in an s3guard command, e.g. "create-role-keys", which would
take the existing (full) credentials and optionally: 
>  * ARN of role to adopt
>  * duration
>  * name
>  * restrictions as path to a JSON file or just stdin
>  * output format
>  * whether to use a per-bucket binding for the credentials in the property names generated
>  * MFA secrets
> output formats
> * A JCEKS file (with chosen passwd? For better hive use: append/replace entries in existing
file); saved through the hadoop FS APIs to HDFS, file:// or elsewhere
> * hadoop config XML
> * spark properties
> The goal here is to have a workflow where you can generate role credentials to use for
a limited time, store them in a JCEKS file and then share them in your jobs. This can be for:
Jenkins, Oozie, build files, ..

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message