sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-327) Mixed update/insert export support for OracleManager
Date Wed, 07 Sep 2011 04:33:12 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098589#comment-13098589
] 

jiraposter@reviews.apache.org commented on SQOOP-327:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1717/#review1779
-----------------------------------------------------------


Changes look good. One question/suggestion below.


src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java
<https://reviews.apache.org/r/1717/#comment4061>

    How about extending from ExportBatchOutputFormat? That would likely work as well unless
I am missing something.


- Arvind


On 2011-09-06 18:09:15, Bilung Lee wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1717/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-09-06 18:09:15)
bq.  
bq.  
bq.  Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  A new option is introduced to allow update records if they exist in table already or
to insert records if they do not exist yet.
bq.  
bq.  
bq.  This addresses bug SQOOP-327.
bq.      https://issues.apache.org/jira/browse/SQOOP-327
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/docs/man/sqoop-export.txt 50052cc 
bq.    src/docs/user/export.txt 4401c26 
bq.    src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc 
bq.    src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b 
bq.    src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d 
bq.    src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION 
bq.    src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION 
bq.    src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 
bq.    src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 
bq.    src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb 
bq.    src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 
bq.  
bq.  Diff: https://reviews.apache.org/r/1717/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Bilung
bq.  
bq.



> Mixed update/insert export support for OracleManager
> ----------------------------------------------------
>
>                 Key: SQOOP-327
>                 URL: https://issues.apache.org/jira/browse/SQOOP-327
>             Project: Sqoop
>          Issue Type: New Feature
>            Reporter: Bilung Lee
>            Assignee: Bilung Lee
>
> Currently Sqoop export job runs in insert mode (default) or update mode (with --update-key
option). When in insert mode, all data are inserted and when in update mode, all data are
updated in existing table. This leaves out the use case where some data may need to be updated
while the rest needs to be inserted.
> It also causes problems
> - When in insert mode, new data may cause constraint violations if they exist already.
> - When in update mode, it could result in silent dropping of records that do not match
the update key.
> The idea is to introduce a new "upsert" mode to update records if they exist in table
already or to insert records if they do not exist yet.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message