sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szabolcs Vasas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3016) warehouse-dir or target-dir is not being honored
Date Thu, 02 Mar 2017 12:44:45 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892176#comment-15892176
] 

Szabolcs Vasas commented on SQOOP-3016:
---------------------------------------

Hi [~ellieeee],

I think the problem is that Sqoop tries to create a MapReduce staging directory under /user
but it fails because the necessary permissions are missing.
You can find more info on MR staging directories here:
https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_ig_yarn_cluster_deploy.html#topic_11_4_5

I have managed to reproduce this issue on my cluster and it was working by overriding yarn.app.mapreduce.am.staging-dir
value:

sqoop import -Dyarn.app.mapreduce.am.staging-dir=/tmp/sqoopstaging --connect "mydatabase"
--username "sqoop" --password "sqoop" --table "mytable" --delete-target-dir --target-dir "/tmp/mytable"

I hope this helps.

Regards,
Szabolcs

> warehouse-dir or target-dir is not being honored 
> -------------------------------------------------
>
>                 Key: SQOOP-3016
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3016
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/hdfs
>    Affects Versions: 1.4.5
>         Environment: CDH 5.4.7
>            Reporter: Kingsly J
>
> Hi Team,
> In our QA env, we are not having sudo access and not having folder in our username in
"/user" and not having write access to the same. Hence we used --warhouse-dir to direct the
result to the directory for which we are having write access.we tried the below sqoop statement
to load data from Teradata to hdfs,
> sqoop import --connect jdbc:teradata://<ip>/Database=test,LOGMECH=LDAP --username
abc -P --table tbl1 --split-by tbl_cd --warehouse-dir /projects/Sqoop_Test
> but getting the below error,
> ERROR tool.ImportTool: Encountered IOException running import job: com.teradata.connector.common.exception.ConnectorException:
org.apache.hadoop.security.AccessControlException: Permission denied: user=abc, access=WRITE,
inode="/user":hdfs:supergroup:drwxr-xr-x
> It is still trying to write to "/user" path. we have tried with --target-dir parameter
also, and still getting the same error.
> Please guide us on the same.
> Regards,
> Kingsly J



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message