sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3396) Add parquet numeric support for Parquet in Hive import
Date Fri, 30 Nov 2018 15:17:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704895#comment-16704895
] 

ASF GitHub Bot commented on SQOOP-3396:
---------------------------------------

Github user fszabo2 commented on a diff in the pull request:

    https://github.com/apache/sqoop/pull/60#discussion_r237894381
  
    --- Diff: src/java/org/apache/sqoop/hive/HiveTypes.java ---
    @@ -37,16 +42,28 @@
       private static final String HIVE_TYPE_STRING = "STRING";
       private static final String HIVE_TYPE_BOOLEAN = "BOOLEAN";
       private static final String HIVE_TYPE_BINARY = "BINARY";
    +  private static final String HIVE_TYPE_DECIMAL = "DECIMAL";
     
       public static final Log LOG = LogFactory.getLog(HiveTypes.class.getName());
     
       private HiveTypes() { }
     
    +
    +  public static String toHiveType(int sqlType, SqoopOptions options) {
    +
    +    if (options.getConf().getBoolean(ConfigurationConstants.PROP_ENABLE_PARQUET_LOGICAL_TYPE_DECIMAL,
false)
    +        && (sqlType == Types.NUMERIC || sqlType == Types.DECIMAL)){
    +      return HIVE_TYPE_DECIMAL;
    +    }
    +    return toHiveType(sqlType);
    +  }
    +
    +
       /**
        * Given JDBC SQL types coming from another database, what is the best
        * mapping to a Hive-specific type?
        */
    -  public static String toHiveType(int sqlType) {
    +  private static String toHiveType(int sqlType) {
    --- End diff --
    
    Actually, there is a method that defines extra mappings for hive types in OraOop:
    org.apache.sqoop.manager.oracle.OraOopConnManager#toHiveType 
    
    So we need to keep the "null" return value here, in order to allow for that to work. 
    
    I think it still makes sense to log a warning though.
    
    On the long run: it might make sense to refactor the ora oop connector, but I think it's
out of the scope of this change.


> Add parquet numeric support for Parquet in Hive import
> ------------------------------------------------------
>
>                 Key: SQOOP-3396
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3396
>             Project: Sqoop
>          Issue Type: Sub-task
>            Reporter: Fero Szabo
>            Assignee: Fero Szabo
>            Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message