sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1306) Allow Sqoop to move files from different FileSystems on incremental import
Date Tue, 15 Apr 2014 20:54:18 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13970050#comment-13970050
] 

Jarek Jarcec Cecho commented on SQOOP-1306:
-------------------------------------------

"Move" is only a metadata operation and hence it's indeed required that they are in the same
filesystem. Otherwise you would need copy or distcopy to make the "move". This limitation
is imposed by HDFS, not by Sqoop.

> Allow Sqoop to move files from different FileSystems on incremental import
> --------------------------------------------------------------------------
>
>                 Key: SQOOP-1306
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1306
>             Project: Sqoop
>          Issue Type: Improvement
>    Affects Versions: 1.4.4
>            Reporter: Rodrigo Matihara
>
> Sqoop doesn't allow to move files from different FileSystems executing an import (--append
or --incremental).
> Trying to import using a local temp-dir and a S3 target-dir, the operation is complete,
however the file is not created in S3 bucket and this warning is raised:
> WARN  - Cannot append files to target dir; no such directory: _sqoop/15151724000000436_31417_localhost.localdomain_<tablename>
> Looking into on source, I found that AppendUtils.java considers that tempDir and targetDir
are at the same Filesystem.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message