sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-1656) SQOOP2: Support for "merge" with different changing dimension types
Date Fri, 07 Nov 2014 03:55:33 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Veena Basavaraj updated SQOOP-1656:
-----------------------------------
    Fix Version/s: 1.99.5

> SQOOP2: Support for "merge" with different changing dimension types
> -------------------------------------------------------------------
>
>                 Key: SQOOP-1656
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1656
>             Project: Sqoop
>          Issue Type: Wish
>            Reporter: Gwen Shapira
>             Fix For: 1.99.5
>
>
> Our current "incremental" design is for "append" only.
> However, we do plan on adding "merge" capabilities somewhere in the future. Maybe.
> Sqoop1 merges by overwriting existing rows with their newer versions.
> But for DWH dimensions, there are other ways to merge:
> http://en.wikipedia.org/wiki/Slowly_changing_dimension
> For example, preserving both versions and adding "start date" and "end date" for each.
> ETL tools can handle these situations. Will be cool if Sqoop2 will be able to do it too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message