sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3330) Sqoop --append does not work with -Dmapreduce.output.basename
Date Fri, 13 Jul 2018 07:36:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542660#comment-16542660
] 

ASF subversion and git services commented on SQOOP-3330:
--------------------------------------------------------

Commit a6bedca4b272173292ab3aeadedea7283aacd16e in sqoop's branch refs/heads/trunk from [~vasas]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=a6bedca ]

SQOOP-3330: Sqoop --append does not work with -Dmapreduce.output.basename

(Eric Lin via Szabolcs Vasas)


> Sqoop --append does not work with -Dmapreduce.output.basename
> -------------------------------------------------------------
>
>                 Key: SQOOP-3330
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3330
>             Project: Sqoop
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 1.4.7
>            Reporter: Eric Lin
>            Assignee: Eric Lin
>            Priority: Major
>         Attachments: SQOOP-3330-1.patch, SQOOP-3330-2.patch, SQOOP-3330.3.patch
>
>
> When adding --append to Sqoop directory import with -Dmapreduce.output.basename, all
files will be ignored, which end up nothing being imported. See below DEBUG output:
> {code}
> sqoop import -Dmapreduce.output.basename="eric-test" --connect jdbc:mysql://mysql-host.com/test
--username root --password 'root' --table test --target-dir /tmp/ericlin-test/sqoop/test --fields-terminated-by
'\t' --verbose --append
> 18/05/28 22:24:44 INFO util.AppendUtils: Appending to directory test
> 18/05/28 22:24:44 DEBUG util.AppendUtils: Filename: _SUCCESS ignored
> 18/05/28 22:24:44 DEBUG util.AppendUtils: Filename: eric-test-m-00000 ignored
> 18/05/28 22:24:44 DEBUG util.AppendUtils: Filename: eric-test-m-00001 ignored
> 18/05/28 22:24:44 DEBUG util.AppendUtils: Filename: eric-test-m-00002 ignored
> 18/05/28 22:24:44 DEBUG util.AppendUtils: Deleting temporary folder 14935e396acc4ea7b9a6236c66064c9b_test
> {code}
> This is due to AppendUtils only recognizes file name starts with "part.*-([0-9]"
> https://github.com/apache/sqoop/blob/branch-1.4.7/src/java/org/apache/sqoop/util/AppendUtils.java#L46



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message