sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Artem Ervits" <are9...@nyp.org>
Subject RE: SQOOP 1.3.0 questions using SQL Server
Date Fri, 26 Oct 2012 21:09:56 GMT
I have Sqoop 1.4.2 installed 

Running sqoop version returns

Sqoop 1.4.2
git commit id
Compiled by ag on Tue Aug 14 17:37:19 IST 2012

When I tried to run a sqoop import-all-tables I got "Unknown SQL Data type: -9". From the
discussion below, nvarchar should be supported in this version of sqoop, please advise?

Thank you.

-----Original Message-----
From: Jarek Jarcec Cecho [mailto:jarcec@apache.org] 
Sent: Monday, October 15, 2012 4:34 PM
To: user@sqoop.apache.org
Cc: Marc Sturm
Subject: Re: SQOOP 1.3.0 questions using SQL Server

Hi Marc,
please find my answers inline:

On Mon, Oct 15, 2012 at 07:25:28PM +0000, Artem Ervits wrote:
> I installed SQOOP 1.4.2 and was able to compress and then uncompress the data. I used
the bin.tar.gz installer.

Great!

> 
> 1.       I did try to use the git repo to compile the sqoop jar and was able to compile
it no problem which resulted in 1.4.3 snapshot jar using command ant jar-all. I noticed that
 all other jars like avro are not present in the build or lib directory. How may I go about
downloading source and compiling it to include avro, etc?

If you need to build entire distribution (like the one that is offered in download page) including
(almost) all dependency jars you need to run command "ant -Dhadoopversion=$hadoopVersion tar",
where $hadoopVersion is your hadoop version. Supported values are 20, 23, 100 and 200.

> 2.       I also tried to use the incremental switch, is this a correct command: "sqoop
import -incremental 'lastmodified'" or does the command look some other way?

Yes that is the intended usage. You might find more information about incremental imports
in Sqoop documentation:

http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_incremental_imports

> Thank you.

Jarcec

> 
> 
> From: Artem Ervits [mailto:are9004@nyp.org]
> Sent: Friday, October 12, 2012 5:17 PM
> To: user@sqoop.apache.org
> Cc: Marc Sturm
> Subject: RE: SQOOP 1.3.0 questions using SQL Server
> 
> Even better!
> 
> Thank you.
> 
> From: Chalcy 
> [mailto:chalcy@gmail.com]<mailto:[mailto:chalcy@gmail.com]>
> Sent: Friday, October 12, 2012 4:18 PM
> To: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
> Subject: Re: SQOOP 1.3.0 questions using SQL Server
> 
> Sqoop 1.4.3 freshly built even works very well with sql server. Just 
> get the sqoop jar and replace it and should work just fine.  I so far 
> never had any issues changing sqoop version :)
> 
> Good luck!
> 
> Thanks,
> Chalcy
> On Fri, Oct 12, 2012 at 2:16 PM, Artem Ervits <are9004@nyp.org<mailto:are9004@nyp.org>>
wrote:
> Actually I was just able to test 3) below and it succeeded, I cast the nvarchar as char.
> 
> Thank you again, I will do 1) and 2) and report back the results.
> 
> -----Original Message-----
> From: Jarek Jarcec Cecho 
> [mailto:jarcec@apache.org<mailto:jarcec@apache.org>]
> Sent: Friday, October 12, 2012 1:51 PM
> To: user@sqoop.apache.org<mailto:user@sqoop.apache.org>
> Subject: Re: SQOOP 1.3.0 questions using SQL Server Hi Artem, Sqoop 
> 1.3.0 is very old release, so I would definitely recommend to upgrade to latest 1.4.2.
Please note that this release include SQOOP-480[1] that fixes some issues for connectors compiled
against 1.3 in 1.4. Sqoop is simple command line utility so you might try to download release
into some temporary folder and verify that everything is working prior upgrading Sqoop system
wise.
> 
> Specifically to your questions
> 
> 1) --incremental
> Usually Sqoop will print out error message first followed by help page, so I would advise
checking few initial lines to see what exactly is wrong.
> 
> 2) --compress
> Please try to reproduce this issue on 1.4.2 and file jira if you'll be able to reproduce
it.
> 
> 3) --as-avrodatafile
> Based on [2] SQL type -9 corresponds to data type NVARCHAR. We've added support for NVARCHAR
in SQOOP-323 [3] that is part of Sqoop since version 1.4.0-incubating - e.g. 1.3.0 version
do not support NVARCHAR type out of the box.
> 
> Jarcec
> 
> Links:
> 1: https://issues.apache.org/jira/browse/SQOOP-480
> 2: 
> http://docs.oracle.com/javase/6/docs/api/constant-values.html#java.sql
> .Types.DECIMAL
> 3: https://issues.apache.org/jira/browse/SQOOP-323
> 
> On Fri, Oct 12, 2012 at 05:29:41PM +0000, Artem Ervits wrote:
> >
> > Hello all,
> >
> > I'm testing a few switches for Sqoop import and I'm having the following problems.
> >
> > When I use the -incremental switch, the command fails and shows me the help page.
> > When I use the -compress switch, the command works but when I try to uncompress
the results, it says gzip undefined compression code. I also tried to explicitly state the
codec and it would still not append the compression extension to the files nor am I able to
uncompress the data.
> > When I use the -as-avrodatafile I get ERROR tool.ImportTool: 
> > Imported Failed: Cannot convert SQL type -9
> >
> > Any ideas? I am not sure if upgrading Sqoop will fix it because SQL Server connector
specifically required a 1.3.0 release.
> >
> > Thanks.
> >
> > Artem Ervits
> > Data Analyst
> > New York Presbyterian Hospital
> >
> >
> >
> > --------------------
> >
> > This electronic message is intended to be for the use only of the named recipient,
and may contain information that is confidential or privileged.  If you are not the intended
recipient, you are hereby notified that any disclosure, copying, distribution or use of the
contents of this message is strictly prohibited.  If you have received this message in error
or are not the named recipient, please notify us immediately by contacting the sender at the
electronic mail address noted above, and delete and destroy all copies of this message.  Thank
you.
> >
> >
> >
> >
> > --------------------
> >
> > This electronic message is intended to be for the use only of the named recipient,
and may contain information that is confidential or privileged.  If you are not the intended
recipient, you are hereby notified that any disclosure, copying, distribution or use of the
contents of this message is strictly prohibited.  If you have received this message in error
or are not the named recipient, please notify us immediately by contacting the sender at the
electronic mail address noted above, and delete and destroy all copies of this message.  Thank
you.
> >
> >
> >
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and
may contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and
may contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 
> 
> 
> --------------------
> 
> Confidential Information subject to NYP's (and its affiliates') information management
and security policies (http://infonet.nyp.org/QA/HospManual/).
> 
> 
> 
> --------------------
> 
> 
> 
> This electronic message is intended to be for the use only of the named recipient, and
may contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 
> 
> 
> ________________________________
> 
> Confidential Information subject to NYP's (and its affiliates') information management
and security policies (http://infonet.nyp.org/QA/HospitalManual).
> 
> 
> --------------------
> Confidential Information subject to NYP's (and its affiliates') information management
and security policies (http://infonet.nyp.org/QA/HospManual/).
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and
may contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 


--------------------

This electronic message is intended to be for the use only of the named recipient, and may
contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.




--------------------

This electronic message is intended to be for the use only of the named recipient, and may
contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.




Mime
View raw message