sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkat Ranganathan <vranganat...@hortonworks.com>
Subject Re: Sqoop2 setup questions
Date Sat, 05 Apr 2014 23:12:10 GMT
Suhas

Transfer to/from Hive/Hbase (through Hcat) are supported from Sqoop 1.4.4.
  See HCatalog Integration

Venkat


On Fri, Apr 4, 2014 at 10:51 AM, Suhas Satish <suhas.satish@gmail.com>wrote:

> Hi Devin,
> Please see inline.
>
> On Thu, Apr 3, 2014 at 10:52 AM, Devin Suiter RDX <dsuiter@rdx.com> wrote:
>
>> regarding running Sq2 client on the same box as the server, essentially
>> stopping it from being worked with in pseudo-distributed mode?
>>
>
> client and server can be run on the same box. You may be using the wrong
> (old) tar ball release for sqoop2 server.
>
> use this -
> You are using old svn code build and setup.
>
> Check this link to build:
> http://sqoop.apache.org/docs/1.99.3/BuildingSqoop2.html
>
>
> Installation part given in :
> http://sqoop.apache.org/docs/1.99.3/Installation.html
> mvn compile has a bug, use mvn install instead
>
>
>
>
>
>> Or does something stand out as a problem with my config/is there some
>> debug I can pull out to glean more insight into why the service isn't
>> listening on that port, even though it appears to be?
>>
> Check tomcat's connector port in server.xml
> it should look something like this -
>
> <Connector port="${sqoop.http.port}" protocol="HTTP/1.1"
>
> If you find that its hardcoded to a default port number instead, then that
> is your problem, the port is not configurable. Change that line to the
> above so it recognizes the -D option given to start the sqoop2 server
> process.
>
>
>
>
>>
>> Thanks for any assistance you can give. I'd like to avoid spinning up a
>> fully-distributed VM cluster to test Sqoop2 jobs on, if possible, and my
>> real cluster is indisposed at the moment...
>>
>
> You can run sqoop 1 and sqoop2 server on different nodes in the cluster
> simultaneously, but not on the same box.
>
> So that will help you a lot in migration, as sqoop2 server still has many
> limitations as compared below in cdh documentation -
>
>
> Feature Differences - Sqoop and Sqoop 2
>    *Note*:
>
> *Moving from Apache Sqoop to Sqoop 2:* Sqoop 2 is essentially the future
> of the Apache Sqoop project. However, since Sqoop 2 currently lacks some of
> the features of Sqoop, Cloudera recommends you use Sqoop 2 only if it
> contains all the features required for your use case, otherwise, continue
> to use Sqoop.
>  Feature Sqoop Sqoop 2Connectors for all major RDBMS Supported.
>
> Not supported.
>
> *Workaround*: Use the generic JDBC Connector which has been tested on the
> following databases: Microsoft SQL Server, PostgreSQL, MySQL and Oracle.
>
> This connector should work on any other JDBC compliant database. However,
> performance might not be comparable to that of specialized connectors in
> Sqoop.
> Kerberos Security IntegrationSupported.
>
> Not supported.
> Encryption of Stored PasswordsNot supported. No workaround. Supported
> using Derby's on-disk encryption<http://db.apache.org/derby/docs/10.2/devguide/cdevcsecure24366.html>
> .
>
> *Disclaimer:* Although expected to work in the current version of Sqoop
> 2, this configuration has not been verified.
> Data transfer from RDBMS to Hive or HBaseSupported.
>
> Not supported.
> *Workaround:* Follow this two-step approach.
>
>    1. Import data from RDBMS into HDFS (either as a text or sequence file)
>    2. Export to Hive or HBase using Sqoop 2
>
> Data transfer from Hive or HBase to RDBMSNot supported.
> *Workaround:* Follow this two-step approach.
>
>    1. Import data from Hive or HBase into HDFS (either as a text or
>    sequence file)
>    2. Export to RDBMS using Sqoop 2
>
> Not supported.
>
> Follow the same workaround as for Sqoop.
>
>
>
> Thanks,
> Suhas.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message