sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec" <jar...@jarcec.net>
Subject Review Request: SQOOP-331 Support boundary query on the command line
Date Sat, 17 Sep 2011 08:12:18 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for Sqoop.


I've incorporated all Arvind's suggestions (hopefully :-)).

This addresses bug SQOOP-331.


  /src/docs/man/import-args.txt 1171925 
  /src/docs/user/import.txt 1171925 
  /src/java/com/cloudera/sqoop/SqoopOptions.java 1171925 
  /src/java/com/cloudera/sqoop/manager/SqlManager.java 1171925 
  /src/java/com/cloudera/sqoop/mapreduce/DataDrivenImportJob.java 1171925 
  /src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 1171925 
  /src/java/com/cloudera/sqoop/tool/ImportTool.java 1171925 
  /src/test/com/cloudera/sqoop/TestSqoopOptions.java 1171925 

Diff: https://reviews.apache.org/r/1946/diff


I'm still having troubles to create meaningful tests for this patch. I've came up with two
different approaches, but I wasn't able to get running either of them:

1) Use boundary query for limiting import data (like "select 1, 2"). This is totally wrong
usage of this parameter, but I was thinking that It might be fine for the testing purpose.
Unfortunately underlying code is using this query only in case that is creating more than
one map task and I was not able to forced it create more than one. Which make sense because
the -m parameter is also only a hint.

2) Parse logs. Fortunately class responsible for creating splits is printing used boundary
query, so there is possibility to parse those logs and look for used boundary query. But I'm
not sure how this can be done in proper fashion.

Any ideas will be welcomed.




  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message