sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1810) GenericJdbcConnector : ToJobConfiguration and Loader enhancements for Incremental write ( INSERT or UPDATE)
Date Mon, 23 Mar 2015 20:51:53 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376343#comment-14376343
] 

Veena Basavaraj edited comment on SQOOP-1810 at 3/23/15 8:51 PM:
-----------------------------------------------------------------

The patch contains the following

1. A new IncrementalWriteConfig similar to IncrementalRead, it has a enum that expects a value
for type of writes to happen
a. INSERT ( default)
b. UPSERT ( insert or update ), for new rows will be inserted
c. UPDAT ( only updates, new rows are left)

Not in the patch
d. DELETE ( in future even support deleting records if need be ) 

2. Changes to the Loader to use the relevant JDBC statements, such as INSERT or UPDATE based
on the primary key

3. The integration test to assert the insert / upsert happened


The current patch (version 1) uploaded does not have an integration test, works manually,
if this approach is acceptable, I will follow up with the full patch with integration tests
 and also support UPSERT, which should be along the same lines, but does a SELECT before updating
or inserting. The code in the JDBCToInitializer might need some refactor to cleanly support
this, since there is a lot of overlap between the insert and update case.



was (Author: vybs):
The patch contains the following

1. A new IncrementalWriteConfig similar to IncrementalRead, it has a enum that expects a value
for type of writes to happen
a. INSERT ( default)
b. UPSERT ( insert or update ), for new rows will be inserted
c. UPDAT ( only updates, new rows are left)

Not in the patch
d. DELETE ( in future even support deleting records if need be ) 

2. Changes to the Loader to use the relevant JDBC statements, such as INSERT or UPDATE based
on the primary key

3. The integration test to assert the insert / upsert happened


> GenericJdbcConnector : ToJobConfiguration and Loader enhancements for Incremental write
( INSERT or UPDATE)
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-1810
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1810
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 2.0.0
>
>
> This feature is a POC to make JDBC Connector support Incremental/Delta read and writes



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message