sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suhas Satish <suhas.sat...@gmail.com>
Subject Re: Sqoop2 setup questions
Date Fri, 04 Apr 2014 17:53:20 GMT
Also check sqoop.log for some helpful hints to troubleshoot your errors.


Cheers,
Suhas.


On Fri, Apr 4, 2014 at 10:51 AM, Suhas Satish <suhas.satish@gmail.com>wrote:

> Hi Devin,
> Please see inline.
>
> On Thu, Apr 3, 2014 at 10:52 AM, Devin Suiter RDX <dsuiter@rdx.com> wrote:
>
>> regarding running Sq2 client on the same box as the server, essentially
>> stopping it from being worked with in pseudo-distributed mode?
>>
>
> client and server can be run on the same box. You may be using the wrong
> (old) tar ball release for sqoop2 server.
>
> use this -
> You are using old svn code build and setup.
>
> Check this link to build:
> http://sqoop.apache.org/docs/1.99.3/BuildingSqoop2.html
>
>
> Installation part given in :
> http://sqoop.apache.org/docs/1.99.3/Installation.html
> mvn compile has a bug, use mvn install instead
>
>
>
>
>
>> Or does something stand out as a problem with my config/is there some
>> debug I can pull out to glean more insight into why the service isn't
>> listening on that port, even though it appears to be?
>>
> Check tomcat's connector port in server.xml
> it should look something like this -
>
> <Connector port="${sqoop.http.port}" protocol="HTTP/1.1"
>
> If you find that its hardcoded to a default port number instead, then that
> is your problem, the port is not configurable. Change that line to the
> above so it recognizes the -D option given to start the sqoop2 server
> process.
>
>
>
>
>>
>> Thanks for any assistance you can give. I'd like to avoid spinning up a
>> fully-distributed VM cluster to test Sqoop2 jobs on, if possible, and my
>> real cluster is indisposed at the moment...
>>
>
> You can run sqoop 1 and sqoop2 server on different nodes in the cluster
> simultaneously, but not on the same box.
>
> So that will help you a lot in migration, as sqoop2 server still has many
> limitations as compared below in cdh documentation -
>
>
> Feature Differences - Sqoop and Sqoop 2
>    *Note*:
>
> *Moving from Apache Sqoop to Sqoop 2:* Sqoop 2 is essentially the future
> of the Apache Sqoop project. However, since Sqoop 2 currently lacks some of
> the features of Sqoop, Cloudera recommends you use Sqoop 2 only if it
> contains all the features required for your use case, otherwise, continue
> to use Sqoop.
>  Feature Sqoop Sqoop 2Connectors for all major RDBMS Supported.
>
> Not supported.
>
> *Workaround*: Use the generic JDBC Connector which has been tested on the
> following databases: Microsoft SQL Server, PostgreSQL, MySQL and Oracle.
>
> This connector should work on any other JDBC compliant database. However,
> performance might not be comparable to that of specialized connectors in
> Sqoop.
> Kerberos Security IntegrationSupported.
>
> Not supported.
> Encryption of Stored PasswordsNot supported. No workaround. Supported
> using Derby's on-disk encryption<http://db.apache.org/derby/docs/10.2/devguide/cdevcsecure24366.html>
> .
>
> *Disclaimer:* Although expected to work in the current version of Sqoop
> 2, this configuration has not been verified.
> Data transfer from RDBMS to Hive or HBaseSupported.
>
> Not supported.
> *Workaround:* Follow this two-step approach.
>
>    1. Import data from RDBMS into HDFS (either as a text or sequence file)
>    2. Export to Hive or HBase using Sqoop 2
>
> Data transfer from Hive or HBase to RDBMSNot supported.
> *Workaround:* Follow this two-step approach.
>
>    1. Import data from Hive or HBase into HDFS (either as a text or
>    sequence file)
>    2. Export to RDBMS using Sqoop 2
>
> Not supported.
>
> Follow the same workaround as for Sqoop.
>
>
>
> Thanks,
> Suhas.
>

Mime
View raw message