sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop --incremental lastmodified --last-value
Date Fri, 08 Aug 2014 21:47:40 GMT
Hey there,

Sqoop returns the current timestamp. If you look closely at the bounding
query, it uses whatever is supplied in "--last-value" as the lower boundary
and the current system time as the upper boundary:

sqoop import --connect "jdbc:mysql:///test" --table lastmod --incremental
lastmodified --check-column created --last-value "2014-08-08 11:19:41.0" ...

14/08/08 14:45:10 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
SELECT MIN(`id`), MAX(`id`) FROM `lastmod` WHERE ( `created` >= '2014-08-08
11:19:41.0' AND `created` < '2014-08-08 14:45:08.0' )

In the above example, 2014-08-08 11:19:41.0 is the lower boundary or the
supplied "--last-value". Also, 2014-08-08 14:45:08.0 was the current system
time at the time of running the script.
-Abe


On Fri, Aug 8, 2014 at 2:18 PM, Vikash Talanki -X (vtalanki - INFOSYS
LIMITED at Cisco) <vtalanki@cisco.com> wrote:

>  Hi All,
>
>
>
> I am using *--incremental lastmodified* in sqoop to get updated data and
> everything seems to be good except the *--last-value* that sqoop prints
> in output after successful import.
>
> Need more insight into what value does sqoop print in output
> for--last-value and how it approach that value.
>
>
>
> Sqoop output after successful import:
>
> [image: Screen capture]
>
>
>
> Maximum value in that column
>
> [image: Screen capture]
>
>
>
> I initially thought it prints the maximum of *--check-column* value
> (max(LAST_UPDATE_DATE) column in my case) but that doesn’t happen.
>
>
>
> Please help me understand this.
>
>
>
>
>
> [image: http://www.cisco.com/web/europe/images/email/signature/logo05.jpg]
>
> *Vikash Talanki*
> Engineer - Software
> vtalanki@cisco.com
> Phone: *+1 (408)838 4078 <%2B1%20%28408%29838%204078>*
>
> *Cisco Systems Limited*
> SJ-J 3
> 255 W Tasman Dr
> San Jose
> CA – 95134
> United States
> Cisco.com <http://www.cisco.com/>
>
>
>
> [image: Think before you print.]Think before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/index.html
>
>
>
>
>

Mime
View raw message