sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anna Szonyi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3136) Sqoop should work well with not default file systems
Date Fri, 24 Feb 2017 10:28:44 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15882404#comment-15882404

Anna Szonyi commented on SQOOP-3136:

Hi [~yalovyyi] [~maugli],

Thanks for the patch, this is super useful! However I am/HUDSON is having issues running the
tests - /src/test/org/apache/sqoop/util/TestFileSystemUtil.java:36: error: cannot find symbol
    conf.set("fs.my.impl", MyFileSystem.class.getTypeName());
As far as I know [this method|https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getTypeName--]
was introduced in Java 1.8 and currently Sqoop supports/is built with/for 1.7 which is causing
the above issue.

Could you please look into this?


> Sqoop should work well with not default file systems
> ----------------------------------------------------
>                 Key: SQOOP-3136
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3136
>             Project: Sqoop
>          Issue Type: Improvement
>          Components: connectors/hdfs
>    Affects Versions: 1.4.5
>            Reporter: Illya Yalovyy
>            Assignee: Illya Yalovyy
>         Attachments: SQOOP-3136.patch
> Currently Sqoop assumes default file system when it comes to IO operations. It makes
it hard to use other FileSystem implementations as source or destination. Here is an example:
> {code}
> sqoop import --connect <JDBC CONNECTION> --table table1 --driver <JDBC DRIVER>
--username root --password **** --delete-target-dir --target-dir s3a://some-bucket/tmp/sqoop
> ...
> 17/02/15 19:16:59 ERROR tool.ImportTool: Imported Failed: Wrong FS: s3a://some-bucket/tmp/sqoop,
expected: hdfs://<DNS>:8020
> {code}

This message was sent by Atlassian JIRA

View raw message