sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Artem Ervits" <are9...@nyp.org>
Subject RE: question about -target-dir
Date Thu, 07 Mar 2013 16:28:49 GMT
It has something to do with my ssh connection. Ssh session terminates before the job is finished.
Since $(date +%Y%m%d%H%M%S) happens as the last step, and if session is dead by then, it has
no way of knowing the final target dir. Is there a way to avoid it? I am using screen tool
to start ssh session, then use the sqoop command and then detach the session, that way session
is never terminated.

-----Original Message-----
From: Jarek Jarcec Cecho [mailto:jarcec@apache.org] 
Sent: Saturday, February 23, 2013 1:15 PM
To: user@sqoop.apache.org
Subject: Re: question about -target-dir

Hi Artem,
would you mind sharing entire Sqoop log and associated mapreduce job configuration object?
The later can be retrieved in form of XML file from JobTracker web UI.

Jarcec

On Fri, Feb 22, 2013 at 04:07:46AM +0000, Artem Ervits wrote:
> Hello all,
> 
> I'm running Sqoop 1.4.2. I'm running into a weird issue when I execute the following
command:
> 
> sqoop job --exec IncrementalLoad -- -target-dir /PROD/SQOOP/$(date +%Y%m%d%H%M%S)
> 
> The table I'm loading has 31 million rows. When job completes, the target-dir still remains
to be something like this: _sqoop/21225255558v_tblNameSqoop and not the designated directory.
It works as expected on smaller datasets but for some reason on this much data, it doesn't.
Also, I just added two nodes to the 4-node clusters, why does Sqoop by default chooses to
use 4 input paths and not 6?
> 
> Thank you.
> 
> Artem Ervits
> Artem@nyp.org<mailto:Artem@nyp.org>
> New York Presbyterian Hospital
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and
may contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and
may contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 


--------------------

This electronic message is intended to be for the use only of the named recipient, and may
contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.




--------------------

This electronic message is intended to be for the use only of the named recipient, and may
contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.




Mime
View raw message