sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ruslan Dautkhanov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3022) sqoop export for Oracle generates tremendous amounts of redo logs
Date Thu, 06 Oct 2016 21:24:20 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553213#comment-15553213
] 

Ruslan Dautkhanov commented on SQOOP-3022:
------------------------------------------

I found this comment in source code

https://github.com/apache/sqoop/blob/7c1754270ff21f533088b946c873321f890da791/src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatInsert.java#L64

{quote}
      // NB: "Direct inserts" cannot utilize APPEND_VALUES, otherwise Oracle
      // will serialize
      // the N mappers, causing a lot of lock contention.
{quote}

I don't think it's true if you don't have neither indexes nor FK constraints on target table
in Oracle.
See https://docs.oracle.com/cd/B19306_01/server.102/b14215/ldr_modes.htm :
{quote}
- Neither local or global indexes can be maintained by the load.
- Referential integrity and CHECK constraints must be disabled.
..
{quote}
If both of the above are satisfied, Oracle can do direct path insert from multiple sessions
(multiple sqoop export mappers' connections).


> sqoop export for Oracle generates tremendous amounts of redo logs
> -----------------------------------------------------------------
>
>                 Key: SQOOP-3022
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3022
>             Project: Sqoop
>          Issue Type: Bug
>          Components: codegen, connectors, connectors/oracle
>    Affects Versions: 1.4.3, 1.4.4, 1.4.5, 1.4.6
>            Reporter: Ruslan Dautkhanov
>              Labels: export, oracle
>
> Sqoop export for Oracle generates tremendous amounts of redo logs (comparable to export
size or more).
> We have put target tables in nologgin mode, but Oracle will still generate redo logs
unless +APPEND Oracle insert hint is used.
> See https://oracle-base.com/articles/misc/append-hint for examples.
> Please add an option for sqoop to generate insert statements in Oracle with APPEND statement.
Our databases are swamped with redo/archived logs whenever we sqoop data to them. This is
easily avoidable. And from business prospective sqooping to staging tables in nologgin mode
is totally fine.
> Thank you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message