sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devin Suiter RDX <dsui...@rdx.com>
Subject Edit/change saved job without losing incremental history?
Date Fri, 31 Jan 2014 02:52:06 GMT

I am exploring Sqoop, and wondering about a hypothetical problem. What
happens if someone needs to change a saved incremental append job in Sqoop
1? Say for example the connection string needed to be changed because of a
network change or something. Shouldn't do that to your cluster, I know,
but, things happen.

>From what I have seen, there isn't any way to edit an in-place job...only
to delete it and create a new one. If you have to do that, I'm guessing the
metastore will drop the history of the incremental append last value. If
you build a new incremental append job, that will bring everything over
from the beginning, which may not be good to do. If you run an incremental
import specifying the "from last value" parameter, that won't be a good for
a saved job, will it? Wouldn't it just begin from that value every time?

Is there a way to edit saved job parameters in the metastore directly so
you can update the connection string or inject the "last value" value in
the new job history after you create it but before running it the first
time? Or is something saved by job name?

Just wondering if there is a solution here that isn't really apparent in
the documentation...

*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com

View raw message