sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "C Scyphers (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SQOOP-3281) Support for Hive UDFs on import
Date Mon, 29 Jan 2018 15:58:00 GMT
C Scyphers created SQOOP-3281:
---------------------------------

             Summary: Support for Hive UDFs on import
                 Key: SQOOP-3281
                 URL: https://issues.apache.org/jira/browse/SQOOP-3281
             Project: Sqoop
          Issue Type: Improvement
          Components: hive-integration
    Affects Versions: 1.4.6
            Reporter: C Scyphers


As many companies are using UDF to establish column level encryption during write time, Sqoop
should support applying such a UDF during the write process.  This would be an extension
of the map-column-hive functionality, where the value of the parseColumnMapping would accept
the UDF:

{{sqoop import --verbose --connect "jdbcconnectionstring" --username user --password password
--hive-import --hive-database hiveschematest --map-column-hive "emptest.id=int,emptest.name=varchar(100),emptest.ssn=UDF_ENCRYPT()"
-m 1}}

With this approach, the data does not have to be written to HDFS in the clear.  This functionality
can also be extended to other UDFs (naturally).

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message