sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "C Scyphers (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SQOOP-3281) Support for Hive UDFs on import
Date Mon, 29 Jan 2018 15:58:00 GMT
C Scyphers created SQOOP-3281:

             Summary: Support for Hive UDFs on import
                 Key: SQOOP-3281
                 URL: https://issues.apache.org/jira/browse/SQOOP-3281
             Project: Sqoop
          Issue Type: Improvement
          Components: hive-integration
    Affects Versions: 1.4.6
            Reporter: C Scyphers

As many companies are using UDF to establish column level encryption during write time, Sqoop
should support applying such a UDF during the write process.  This would be an extension
of the map-column-hive functionality, where the value of the parseColumnMapping would accept
the UDF:

{{sqoop import --verbose --connect "jdbcconnectionstring" --username user --password password
--hive-import --hive-database hiveschematest --map-column-hive "emptest.id=int,emptest.name=varchar(100),emptest.ssn=UDF_ENCRYPT()"
-m 1}}

With this approach, the data does not have to be written to HDFS in the clear.  This functionality
can also be extended to other UDFs (naturally).



This message was sent by Atlassian JIRA

View raw message