sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Cecho" <jar...@apache.org>
Subject Re: Review Request 15589: SQOOP-1232 Sqoop2: Provide tooling infrastructure for Sqoop2
Date Mon, 02 Dec 2013 17:13:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15589/
-----------------------------------------------------------

(Updated Dec. 2, 2013, 5:13 p.m.)


Review request for Sqoop.


Changes
-------

Incorporated Abe's feedback.


Bugs: SQOOP-1232
    https://issues.apache.org/jira/browse/SQOOP-1232


Repository: sqoop-sqoop2


Description
-------

I've provided basic infrastructure for the tooling by adding new maven module sqoop-tools
that is depending on the core module. I've also provided example tool called "verify" that
will try to bootstrap all sqoop components to see if they are configured correctly.

I was having difficulties to make run the code with the same environment as the Sqoop 2 server
itself, but without the need to open the Tomcat for outside connections. I finally ended up
using Tool mechanism that is exposed by Tomcat. Sadly this mechanism won't set up the environment
exactly as we need it - common.loader and webapp class loaders are not loaded by this mechanism.
so I've provided a class that will finish configuring the environment and call the usual ToolRunner.
To keep the footprint of this class/jar file to a bare minimum I've placed it into standalone
maven module that do not depend on any Sqoop libraries.


Diffs (updated)
-----

  dist/pom.xml 9186a38b3b9c57a62731f01839a99dc7cad9fc99 
  dist/src/main/bin/sqoop.sh e3ed5ef4bbaba0c3048cb005d4e183e66768117e 
  pom.xml a4bc085ce808db912b6ca516b51c70529bd081e7 
  server/pom.xml a07ecf41c2aa687aa4a2a60367e1d1265610b7e9 
  tomcat/pom.xml PRE-CREATION 
  tomcat/src/main/java/org/apache/sqoop/tomcat/TomcatToolRunner.java PRE-CREATION 
  tools/pom.xml PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/Tool.java PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/ToolRunner.java PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/tool/BuiltinTools.java PRE-CREATION 
  tools/src/main/java/org/apache/sqoop/tools/tool/VerifyTool.java PRE-CREATION 

Diff: https://reviews.apache.org/r/15589/diff/


Testing
-------

I'm currently not attaching any unit tests as this task should be better covered some sort
of integration testing. I did however tested the provided verification tool on a real cluster
both with correct configuration and miss configuration (missing Hadoop dependencies).


Thanks,

Jarek Cecho


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message