sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Konarski <akonar...@gmail.com>
Subject Incremental import using sqoop2
Date Sat, 24 Oct 2015 12:29:40 GMT
I would like to import data from MySQL table into HDFS. I have everything
configured and I am able to create simple job in sqoop-shell that is
copying data. However, I would like to copy each time only new records, but
I am not sure how to achieve this. When I create job there is a parameter
named "check column" and I have columns like ID or eventTimestamp that seem
to be suitable there. However, in such case I should enter also "last
value" then. Do I have to manage this last value by myself and each time
create new job with new "last value"? Why in such case create a job if it
is used only once and then has to be recreated? Is it not possible for
Sqoop to manage this, by storing each time new "last value" and import only
new records? Moreover, why I have this error message when I enter anything
as "last value": "Size of input exceeds allowance for this input field.
Maximal allowed size is -1"?

Best regards,

View raw message