sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yan braun (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-3018) Hadoop MapReduce job submission be done in client user UGI?
Date Fri, 14 Oct 2016 17:42:20 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yan braun updated SQOOP-3018:
-----------------------------
    Attachment: SQOOP-3018.patch

I've made changes to create/update/delete/view the job in proxyuser UGI while  keeping the
proxyuser UGI doAs code in the connectors (which is also used outside of the M/R job).  All
unit tests passed.  I tested it in my environment and it worked fine for me.  Attached the
diff to branch 1.99.7: SQOOP-3018.patch.     [~abrahamfine] and [~jarcec], could you see if
it would work with your use cases as well?

> Hadoop MapReduce job submission be done in client user UGI?
> -----------------------------------------------------------
>
>                 Key: SQOOP-3018
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3018
>             Project: Sqoop
>          Issue Type: New Feature
>          Components: connectors/hdfs
>    Affects Versions: 1.99.7
>            Reporter: Yan Braun
>         Attachments: SQOOP-3018.patch
>
>
> Hdfs Connector read and write to HDFS in client user UGI when proxyUser is enabled. 
But MapReduce job submission is done using Sqoop user UGI, which makes all jobs from different
users run in Sqoop user's hadoop queue  instead of client users' own queue.   
> This is a follow-up JIRA after our discussions with Abraham Fine on whether this will
be on sqoop2 road map in the near future.  Thanks.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message