sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1810) GenericJdbcConnector : ToJobConfiguration and Loader enhancements for Incremental write ( INSERT or UPDATE)
Date Tue, 24 Mar 2015 16:10:54 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378071#comment-14378071
] 

Jarek Jarcec Cecho commented on SQOOP-1810:
-------------------------------------------

Thank you for the preliminary patch [~vybs]. I've took a look and submitted few comments on
the review board, I'm looking forward for the final patch with integration tests.

One question about the {{UPSERT}} mode though. It seems that the current proposal is to issue
single select statement for each row which seems quite expensive operation for the database
(no batching, a lot of queries). In Sqoop 1 we've decided not to do that due to the performance
implications and rather delegate the {{UPSERT}} functionality to database specific connectors
that could take advantage of database natives way of implementing {{UPSERT}} (e.g. to ways
that do not have such performance implications). For example Microsoft SQL Server and Oracle
supports {{MERGE}} statement, MySQL have {{ON DUPLICATE KEY}} clause, ... What are your thoughts
on performance implications of this approach?

> GenericJdbcConnector : ToJobConfiguration and Loader enhancements for Incremental write
( INSERT or UPDATE)
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-1810
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1810
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 2.0.0
>
>
> This feature is a POC to make JDBC Connector support Incremental/Delta read and writes



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message