sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3418) Document decimal support in Hive external import into parquet files
Date Wed, 12 Dec 2018 11:00:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718785#comment-16718785
] 

ASF GitHub Bot commented on SQOOP-3418:
---------------------------------------

Github user fszabo2 commented on a diff in the pull request:

    https://github.com/apache/sqoop/pull/66#discussion_r240968555
  
    --- Diff: src/docs/user/import.txt ---
    @@ -472,36 +472,48 @@ Enabling Logical Types in Avro and Parquet import for numbers
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     
     To enable the use of logical types in Sqoop's avro schema generation,
    -i.e. used during both avro and parquet imports, one has to use the
    -sqoop.avro.logical_types.decimal.enable flag. This is necessary if one
    +i.e. used both during avro and parquet imports, one has to use the
    ++sqoop.avro.logical_types.decimal.enable+ property. This is necessary if one
     wants to store values as decimals in the avro file format.
     
    -Padding number types in avro import
    -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    +In case of a parquet import, one has to use the
    ++sqoop.parquet.logical_types.decimal.enable+ property.
    +
    +Padding number types in avro and parquet import
    ++++++++++++++++++++++++++++++++++++++++++++++++
     
     Certain databases, such as Oracle and Postgres store number and decimal
     values without padding. For example 1.5 in a column declared
    -as NUMBER (20,5) is stored as is in Oracle, while the equivalent
    +as NUMBER (20, 5) is stored as is in Oracle, while the equivalent
     DECIMAL (20, 5) is stored as 1.50000 in an SQL server instance.
    -This leads to a scale mismatch during avro import.
    +This leads to a scale mismatch during the import.
     
    -To avoid this error, one can use the sqoop.avro.decimal_padding.enable flag
    -to turn on padding with 0s. This flag has to be used together with the
    -sqoop.avro.logical_types.decimal.enable flag set to true.
    +To avoid this error, one can use the +sqoop.avro.decimal_padding.enable+
    +property to turn on padding with 0s during. Naturally, this property is used
    --- End diff --
    
    yes!


> Document decimal support in Hive external import into parquet files
> -------------------------------------------------------------------
>
>                 Key: SQOOP-3418
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3418
>             Project: Sqoop
>          Issue Type: Task
>            Reporter: Fero Szabo
>            Assignee: Fero Szabo
>            Priority: Major
>             Fix For: 1.5.0, 3.0.0
>
>
> Remember to note the limitations in Hive i.e. the max scale and precision is 38 and how
it behaves in edge cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message