sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Attila Szabo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3022) sqoop export for Oracle generates tremendous amounts of redo logs
Date Fri, 07 Oct 2016 00:09:20 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553631#comment-15553631
] 

Attila Szabo commented on SQOOP-3022:
-------------------------------------

Hi [~Tagar],

In connection with "ORA-12838: cannot read/modify an object after modifying it in parallel"
you're absolutely right, and if you check the OraOop related sources you could see we're doing
exactly the same what was your guess:
1 transaction, many records. It is scalable, but has to be aware about that when parallel
writing path is open no one else should try to read/modify.

To make it even more scalable OraOop has a solution to create a partitioned table, insert
into those partition tables, and at the end only move the partition under the table with easy
to execute alter statements. It's working quite good.

On the front of INSERT ALL:
Never tried it, have to check it tomorrow.

> sqoop export for Oracle generates tremendous amounts of redo logs
> -----------------------------------------------------------------
>
>                 Key: SQOOP-3022
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3022
>             Project: Sqoop
>          Issue Type: Bug
>          Components: codegen, connectors, connectors/oracle
>    Affects Versions: 1.4.3, 1.4.4, 1.4.5, 1.4.6
>            Reporter: Ruslan Dautkhanov
>              Labels: export, oracle
>
> Sqoop export for Oracle generates tremendous amounts of redo logs (comparable to export
size or more).
> We have put target tables in nologgin mode, but Oracle will still generate redo logs
unless +APPEND Oracle insert hint is used.
> See https://oracle-base.com/articles/misc/append-hint for examples.
> Please add an option for sqoop to generate insert statements in Oracle with APPEND statement.
Our databases are swamped with redo/archived logs whenever we sqoop data to them. This is
easily avoidable. And from business prospective sqooping to staging tables in nologgin mode
is totally fine.
> Thank you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message