sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed El Baz" <ahel...@microsoft.com>
Subject Re: Review Request: Request to review patch for SQOOP-954: Create Sqoop runtime scripts to run Sqoop on Windows
Date Mon, 22 Apr 2013 03:26:47 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated April 22, 2013, 3:26 a.m.)

Review request for Sqoop.


As per the discussions below, updating configure-sqoop script to use the parent directories
of classpath jars rather than using a single jar which references all other jars in its manifest.
For HBase we use HBASE_HOME and HBASE_HOM\lib, rather than invoking hbase -classpath as in
the Linux version.
Removed the BuildJar.ps1 script since we no longer need to construct a jar with all classpath
jars in its manifest.


A patch implementing the Windows version of Sqoop run scripts. The scripts follow the same
logic as there .sh counterparts.
One difference is to create a Jar which references all classpath elements in its Manifest,
and provide that jar as the single jar needed for Sqoop. The reason here is that in some cases
if the number of classpath elements is large, HADOOP_CLASSPATH gets very long which causes
failures in Windows since there is a limit to command lines.
As a workaround, I added a step to wrap all jars in the classpath in a single jar, and then
use that generated jar (this is also done in hadoop for Windows to handle similar issues)
I did this in a utility script "BuildJar" which can be used for other components as well.
This change is specific to Windows scripts, Linux scripts are not affected.

This addresses bug SQOOP-954.

Diffs (updated)

  bin/configure-sqoop.cmd PRE-CREATION 
  bin/sqoop.cmd PRE-CREATION 
  conf/sqoop-env-template.cmd PRE-CREATION 

Diff: https://reviews.apache.org/r/10055/diff/



Ahmed El Baz

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message