sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abraham Elmahrek <...@cloudera.com>
Subject Re: Sqoop --incremental lastmodified --last-value
Date Fri, 08 Aug 2014 22:41:43 GMT
Huh, that's really weird. What is the result of the date command in linux
on that machine?

On Fri, Aug 8, 2014 at 3:16 PM, Vikash Talanki -X (vtalanki - INFOSYS
LIMITED at Cisco) <vtalanki@cisco.com> wrote:

>  Thanks a ton Abraham. Understood the point.
>
>
>
> But in my case its showing current time(upper boundary) in GMT time zone
>
>
>
> Even when I use *-D oracle.sessionTimeZone=America/Los_Angeles* in my
> sqoop command, it is considering the upper boundary in GMT only
>
>
>
> Is there a way to handle this time zone issue, so that I will not loose
> records updated in that 7 hrs difference(GMT – Los Angeles TZ = 7 hours)
> when this is job is executed next time.
>
>
>
> Also please let me know,if considering max(last_update_date) from current
> run for *--last-value* for subsequent run gives me the same result… I
> think it should.
>
>
>
> Thanks,
>
> Vikash Talanki
>
> +1 (408)838-4078
>
>
>
> *From:* Abraham Elmahrek [mailto:abe@cloudera.com]
> *Sent:* Friday, August 08, 2014 2:48 PM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Sqoop --incremental lastmodified --last-value
>
>
>
> Hey there,
>
>
>
> Sqoop returns the current timestamp. If you look closely at the bounding
> query, it uses whatever is supplied in "--last-value" as the lower boundary
> and the current system time as the upper boundary:
>
> sqoop import --connect "jdbc:mysql:///test" --table lastmod --incremental
> lastmodified --check-column created --last-value "2014-08-08 11:19:41.0" ...
>
> 14/08/08 14:45:10 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> SELECT MIN(`id`), MAX(`id`) FROM `lastmod` WHERE ( `created` >= '2014-08-08
> 11:19:41.0' AND `created` < '2014-08-08 14:45:08.0' )
>
> In the above example, 2014-08-08 11:19:41.0 is the lower boundary or the
> supplied "--last-value". Also, 2014-08-08 14:45:08.0 was the current system
> time at the time of running the script.
>
> -Abe
>
>
>
> On Fri, Aug 8, 2014 at 2:18 PM, Vikash Talanki -X (vtalanki - INFOSYS
> LIMITED at Cisco) <vtalanki@cisco.com> wrote:
>
> Hi All,
>
>
>
> I am using *--incremental lastmodified* in sqoop to get updated data and
> everything seems to be good except the *--last-value* that sqoop prints
> in output after successful import.
>
> Need more insight into what value does sqoop print in output
> for--last-value and how it approach that value.
>
>
>
> Sqoop output after successful import:
>
> [image: Screen capture]
>
>
>
> Maximum value in that column
>
> [image: Screen capture]
>
>
>
> I initially thought it prints the maximum of *--check-column* value
> (max(LAST_UPDATE_DATE) column in my case) but that doesn’t happen.
>
>
>
> Please help me understand this.
>
>
>
>
>
> [image: http://www.cisco.com/web/europe/images/email/signature/logo05.jpg]
>
> *Vikash Talanki*
> Engineer - Software
> vtalanki@cisco.com
> Phone: *+1 (408)838 4078 <%2B1%20%28408%29838%204078>*
>
> *Cisco Systems Limited*
> SJ-J 3
> 255 W Tasman Dr
> San Jose
> CA – 95134
> United States
> Cisco.com <http://www.cisco.com/>
>
>
>
> [image: Think before you print.]Think before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/index.html
>
>
>
>
>
>
>

Mime
View raw message