sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3438) Sqoop Import with create hcatalog table for ORC will not work with Hive3 as the table created would be a ACID table and transactional
Date Wed, 05 Jun 2019 08:40:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856496#comment-16856496
] 

ASF subversion and git services commented on SQOOP-3438:
--------------------------------------------------------

Commit be260e3096a6a2710e661c7fe33f6b402ce66476 in sqoop's branch refs/heads/trunk from Denes
Bodo
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=be260e3 ]

SQOOP-3438: Sqoop Import with create hcatalog table for ORC will not work with Hive3 as the
table created would be a ACID table and transactional

(Denes Bodo via Boglarka Egyed)


> Sqoop Import with create hcatalog table for ORC will not work with Hive3 as the table
created would be a ACID table and transactional
> -------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-3438
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3438
>             Project: Sqoop
>          Issue Type: Improvement
>          Components: hive-integration
>    Affects Versions: 1.4.7
>            Reporter: Denes Bodo
>            Assignee: Denes Bodo
>            Priority: Critical
>          Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> PROBLEM: Running a sqoop import command with the option --create-hcatalog-table will
not work due to the following reasons
> When create-hcatalog-table is used it creates the table as a Managed ACID table.
> HCatalog does not support transactional or bucketing table
> So customer who need to create a ORC based table cannot use sqoop to create a ORC based
table which means their existing code where if in case they use sqoop to create these tables
would fail.
> The current workaround is a two step process
> 1. Create the ORC table in hive with the keyword external and set transactional to false
> 2. Then use the sqoop command to load the data into the orc table.
> The request is to add in an extra argument in the sqoop command line to specify that
the table is external (example: --hcatalog-external-table )so we can use the option --hcatalog-storage-stanza
"stored as orc tblproperties (\"transactional\"=\"false\")".
> ----
> Thank you [~mbalakrishnan] for your findings. This ticket is created based on your work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message