sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Foerster <t...@phdata.io>
Subject Re: Getting upper bound in --incremental mode
Date Wed, 19 Jul 2017 19:18:59 GMT
Does `--incremental append` work for you?

> You should specify append mode when importing a table where new rows are continually
being added with increasing row id values

Tony

> On Jul 19, 2017, at 2:02 PM, Jagrut Sharma <jagrutsharma@gmail.com> wrote:
> 
> Hi all - For --incremental mode with 'lastmodified' option, Sqoop (v 1.4.2)
> generates a query like:
> WHERE column >= last_modified_time and column < current_time
> 
> The --last-value is set to the current_time and gets used for the next run.
> 
> Here, the upper bound is always set to the current_time. In some cases,
> this upper bound is required to be taken from the database table column
> itself. So, the query is required of the form:
> WHERE column >= last_modified_time and column < max_time_in_db_table_column
> 
> And the --last-value for next run needs to be set as
> the max_time_in_db_table_column (and not the current_time).
> 
> Is this possible to set while running sqoop?  If no, is there any
> workaround suggested for this?
> 
> Thanks a lot.
> -- 
> Jagrut


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message