sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagrut Sharma <jagrutsha...@gmail.com>
Subject Re: Getting upper bound in --incremental mode
Date Wed, 19 Jul 2017 21:43:58 GMT
Hi Tony - I was under the assumption that append mode will not work for
timestamp column. But I gave it a try after your reply, and it works. And
it gets the upper bound from the database itself. Thanks.

--
Jagrut

On Wed, Jul 19, 2017 at 12:18 PM, Tony Foerster <tony@phdata.io> wrote:

> Does `--incremental append` work for you?
>
> > You should specify append mode when importing a table where new rows are
> continually being added with increasing row id values
>
> Tony
>
> > On Jul 19, 2017, at 2:02 PM, Jagrut Sharma <jagrutsharma@gmail.com>
> wrote:
> >
> > Hi all - For --incremental mode with 'lastmodified' option, Sqoop (v
> 1.4.2)
> > generates a query like:
> > WHERE column >= last_modified_time and column < current_time
> >
> > The --last-value is set to the current_time and gets used for the next
> run.
> >
> > Here, the upper bound is always set to the current_time. In some cases,
> > this upper bound is required to be taken from the database table column
> > itself. So, the query is required of the form:
> > WHERE column >= last_modified_time and column <
> max_time_in_db_table_column
> >
> > And the --last-value for next run needs to be set as
> > the max_time_in_db_table_column (and not the current_time).
> >
> > Is this possible to set while running sqoop?  If no, is there any
> > workaround suggested for this?
> >
> > Thanks a lot.
> > --
> > Jagrut
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message